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FOREWORD 
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ABSTRACT 


The  vast  and  rapid  advancement  in  telecommunications,  computers,  controls 
and  aerospace  science  has  necessitated  major  changes  in  our  basic  understand¬ 
ing  of  the  theory  of  electrical  signals  and  processing  systems.  There  is 
strong  evidence  that  today's  engineer  needs  to  extend  and  to  modernize  his 
analytical  techniques.  The  latest  fundamental,  analytical  approach  for  the 
study  of  signals  and  systems  seems  to  have  its  roots  in  the  mathematics  of 
Functional  Analysis. 

This  report  contains  a  bird’s-eye  view  of  the  elements  of  Hilbert  spaces 
and  their  associated  linear  operators.  The  first  chapter  of  the  report  gives 
an  exposition  of  the  most  essential  properties  of  Hilbert  spaces.  The  second, 
chapter  presents  the  elements  of  linear  operators  acting  on  such  spaces. 

The  report  is  addressed  to  engineers  and  scientists  interested  in  the 
theory  of  signals  and  systems.  The  applications  of  the  theory  will  be  under¬ 
taken  in  a  separate  report. 
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Chapter  I 

EIEMENTARY  HILBERT  SPACE  THEORY 


1-1 .  Introduction 

In  the  report  TR-6 5-399  we  have  discussed  the  finite-dimensional  linear 
space  and  metric  apace  in  some  generality.  The  reader  recalls  that  the  con¬ 
cept  of  distance  between  pairs  of  points  of  the  space,  such  as  the  familiar 
Euclidean  distance,  played  an  important  role  in  our  studies.  In  this  chapter, 
we  wish  to  investigate  a  more  general  type  of  space,  that  is,  metric  spaces  of 
infinite  dimensions. 

A  vector  space  of  finite  dimensions  is  said  to  be  an  n- space  if  it  con¬ 
tains  a  maximum  of  n  linearly  independent  elements.  A  vector  space  of  infinite 
dimensions  in  its  simple  form  is  a  generalization  of  the  n- space  when  the  number 
of  linearly  independent  elements  becomes  arbitrarily  large.  An  understanding 
of  the  concept  of  a  space  with  infinite  dimensions  in  itself  requires  seme  pre¬ 
liminary  preparations.  The  introduction  of  "infinity"  is  accompanied  by  certain 
problems  of  convergence  and  continuity.  Therefore,  our  first  Job  is  to  make  some 
inquiry  about  the  continuity  of  the  metric  associated  with  pairs  of  elements  in 
this  space.  Moreover,  when  the  dimension  of  the  space  is  an  uncountable  number 
(nond< numerably  infinite),  the  structure  of  the  space  becomes  considerably  more 
complex.  There  lies  a  professional  area  of  mathematics  beyond  our  aim.  A  com¬ 
prehensive  mathematical  study  of  abstract  Hilbert  spaces  requires  certain 
specialized  preparations  beyond  the  acope  of  this  undertaking.  In  view  of  the 
fact  that  Hilbert  space,  Banach  space,  and  function  space  often  appear  in  engineering 
literature,  a  rudimentary  knowledge  of  the  subject  seems  to  be  indispensible  to 
the  engineer.  To  comply  with  this  need,  we  will  give  an  elementary  account  of 
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the  generalization  of  the  concept  of  a  finite-dimensional  Euclidean  space 
to  the  cuse  where  the  dimension  of  the  space  may  become  countably  infinite . 
We  will  limit  ourselves  to  what  may  be  hopefully  termed,  in  the  words  of 
mathematician  P.  R.  Halmos  ,  "a  glimpse  into  Hilbert  space''. 


The  presence  of  a  metric  on  the  elements  of  X  allows  us  to  talk  naturally  about 
convergence,  and  continuity  in  the  sense  of  the  metric. 

This  section  deals  with  the  general  concept  of  continuity  in  a  metric 
space.  For  the  sake  of  brevity,  we  merely  restrict  ourselves  to  the  most  per¬ 
tinent  definitions  and  basic  theorems.  For  our  limited  purposes  the  definitions 
relevant  to  metric  spaces  may  be  considered  as  generalization) of  the  alike  familiar 
concepts  of  Euclidean  spaces. 


Halmos  studies  in  Lectures  on  Modern  Mathematics.  T.L.  Saaty,  Editor,  Vol.  1, 
John  Wiley  and  Son,  New  York,  pages  1-22,  1965. 
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fThe  words  "element",  "point"  and  "vector"  are  used  synonomouslyj  vectors  are 
underlined  with  a  wavy  line.) 

Open  Sphere  -  An  open  sphere  in  a  metric  space  X,  with  center  at  point  g 
and  radius  r,  is  defined  by 

S(fi,r)  =  {jt  •  x|  d(.g,as)  <  r}  (1-2) 

Closed  Sphere  -  A  closed  sphere  with  center  at  point  g  and  radius  r  is 
defined  by 

S(fi,r)  =  e  X|  d(a,ac)  <  r  j  (1-3) 

The  closed  sphere  includes  the  points  on  its  surface,  i.e.,  points  jc  e  X  such 
that 

<*(£>*)  =  r  (l-1*) 

The  pictorial  representations  are  used  to  facilitate  an  understanding  of  the 
concepts  involved  by  analogy  of  the  Euclidean  space.  They  should  not  be  employed, 
however,  as  substitutes  for  the  formal  definitions.  A  limitation  of  pictorial, 
representation  is,  for  instance,  that  pictures  may  assume  a  variety  of  forms, 
depending  on  the  definition  of  the  distance  function.  For  example,  in  the  space 
of  continuous  functions  C(0, 1]  with  Chebychev  norm  (see  Sec.  h-2 ),  Fig.  1-3  may 
be  envisaged  for  the  open  sphere  d(2-fi  <  £.  (Compare  with  Fig.  1-1.) 
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Neighborhood 

Let  g  be  a  point  of  the  metric  space  Xj  a  set  of  points  in  X  1b  said  to  be  a 
neighborhood  of  a,  denoted  by  N(a),  if  there  exist  a  6  >  0  such  that  N(a)  contains 
the  sphere  S(g, 5),  i.e.: 


N(a)  ^ jjc  e  X|  d(.g,;c)  <  b}  (1-5) 

The  collection  of  all  neighborhoods  of  a  point  is  referred  to  as  the  complete 
or  fundamental  system  of  neighborhoods  of  that  point.  The  characterization  of 
the  continuity  of  a  function  at  a  point  is  based  on  this  concept.  From  this 
definition  it  follows  that  the  open  sphere  S(a,r)  is  a  neighborhood  for  each  of 
its  points.  Geometrically,  the  latter  statement  implies  that  for  any  point  b  in 
S(a,r)  a  number  r'  can  be  found  such  that  S(b,r')  is  contained  in  the  sphere  S(a,r ). 


Fig.  1-1  An  open  sphere  is  a  neighborhood 
for  each  of  its  points. 


Boundedness 


A  non  empty  set  in  a  metric  space  X  is  called  bounded  if  it  is  contained  in 
some  open  sphere  in  X. 

Limit  of  a  sequence 

Let  •  •  •  jbe  a  sequence  of  points  in  a  metric  space  X.  A  point  jg  in  X 

is  said  to  be  the  limit  of  this  sequence,  if  for  any  specified  e  >  0  there  exists 
an  integer  N,  generally  depending  on  €,  such  that  whenever  n  >  N,  then  d(a, )  <  e 
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or  equivalently  lies  in  the  open  sphere  S(a,e).  This  statement,  sometimes, 
is  abbreviated  by  writing 

lim  d(a,a  )  =  0  (1-6) 

n  -» eo  '  n 

Alternatively,  it  is  said  that  the  sequence  converges  to  a 

lim  a  =  a  (1-7) 

„  ~n  — 

n  oo 

Function- continuous  mapping 

Let  X  and  X'  be  metric  spaces  with  distance  functions  d  and  d',  and  let  f  be  a 
mapping  of  X  into  X'.  A  function  of  a  vector  argument  f(x)  defined  over  subset 
D  of  a  metric  space  X  is  said  to  be  a  continuous  mapping  function  at  a  point 
xQ  e  D,  if  for  any  specified  e  >  0  there  exist  a  6  >  0  such  that  whenever 

<1(2»X0)  <  6 

then  in  the  metric  space  of  the  images  X', 

d'(f(as),  f(290))  <  € 

Alternatively,  the  function  is  said  to  be  continuous  at  a  point  a,  if  whenever 
a  is  the  limit  for  a  sequence  of  points  in  X,  then  f(a)  is  the  limit  for  the 
corresponding  sequence  of  image  points  in  the  image  space  X’.  A  mapping  is  said 
to  be  a  continuous  mapping  in  D  if  it  is  continuous  at  each  point  of  D.  In  parti¬ 
cular,  if  f(x)  is  a  numerical  function,  i.e.,  D  is  mapped  into  R'  or  C',  then  we 
have  the  more  familiar  definition  of  a  numerical  function  continuous  at  a  point 
xQ  €  Xj  i.e., 

^(xjXq)  <  6  ==>  |f(x)  -  1(^)1  <  e 
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Fundamental  sequence 

A  sequence  of  points  '*'}  of  a  metric  space  X  is  called  a  fundamental, 

or  a  Cauchy  sequence,  if  for  each  c  >  0  there  is  a  positive  integer  N£  such 

that  d(a  ,a  )<  i_  whenever  n  and  m  >  N  .  It  is  not  difficult  to  see  that  every 
'n  — m  e 

convergent  sequence  is  a  fundamental  sequence .  In  fact,  if  the  sequence  J 
converges  to  f,  then  for  any  specified  e  >  0,  one  can  find  N£  such  that 

d(fn,f )  <  e/2  n  >  N£  (1-8) 

Now  consider  a  point  f  for  m  >  N 
~m  € 

d(f^f  )  <  e/2  (1-9) 

But  due  to  the  triangle  inequality 

dCEm'V  -  d(4i'in)  +  d<£n'£>  <  6  (1-10) 

Thus  a  converging  sequence  is  a  Cauchy  sequence. 

Complete  Space 

A  metric  space  X  is  said  to  be  a  complete  space  if  every  fundamental  sequence 
in  X  converges  to  some  element  in  that  space. 

Separable  Space 

A  thorough  discussion  of  the  separability  property  of  a  metric  space  requires 
some  mathematical  preparations  beyond  the  scope  of  the  present  undertaking.  Since 
the  terms  separability  and  separable  metric  spaces  appear  frequently  in  the 
statements  of  theorems  on  Hilbert  space,  an  introductory  notion  of  this  concept 
is  included.  In  the  space  of  real  numbers  R,  let  us  consider  the  set  of  rational 
numbers;  this  set  has  a  very  important  property.  Every  real  number  x  e  R  can  be 
expressed  in  the  form  of  a  limit  of  a  sequence  of  rational  numbers.  Take,  for 
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instance,  the  number  n-,  the  following  sequence  of  rational  numbers  converges 
to  n: 

3,  3-1,  3-14,  3 • l4l,  3.1415,  3.14159,  ... 

The  set  of  rational  numbers  is  denumerable,  i.e.,  the  rational  numbers  can  be 
put  into  a  one-to-one  correspondence  with  the  set  of  positive  integer.  Further¬ 
more,  the  set  of  rational  numbers  is  dense  in  the  upace  of  real  numbers.  In  other 
words,  between  any  two  distinct  real  numbers  there  is  a  rational  number. 

A  metric  space,  consisting  of  an  infinite  set  of  elements  is  said  to  be 

* 

a  separable  space  if  there  is  a  denumerable  subset  of  elements  dense  in  X: 


(1-11) 


That  is,  for  any  specified  x  e  X  and  e  >  0  there  exists  an  element  xQ  in  the 

o 

above  sequence  such  that 


d(x,2n  )  <  e  (1-12) 

o 

In  the  finite-dimensional  real  Euclidean  space  Rn,  the  set  of  all  points  with 
rational  coordinates  is  a  countable  set  dense  in  Rn.  Therefore,  Rn  is  a 
separable  space.  Same  is  true  for  a  finite-dimensional  complex  space. 


Compact  Set 

A  set  E  of  a  metric  space  X  is  said  to  be  compact  if  every  infinite  sequence  of 


elements  in  E  contains  a  subsequence  which  converges  to  some  x  e  X.  The  require- 


A  more  formal  definition  of  the  term  dense  results  from  the  following.  Let  X 
be  a  metric  space,  and  E  is  a  subset  of  X,  then: 

(1)  x  e  X  is  a  limit  point  of  E  if  every  open  sphei  e  about  x  contains  a  point 
X  4  Z  such  that  y  e  E. 

(2  )  E  is  dense  in  X  if  every  point  of  X  is  either  a  limit  point  of  E,  or  a 
point  of  E  (or  both ) . 


Rudin,  W.,  Principles  of  Mathematical  Analysis,  McGraw-Hill  Book  Co.,  New  York, 
1964,  p.  28. 
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ment  that  a  space  be  compact  is  very  rigid  and  more  restrictive  than  the 
requirement  of  separability  and  completeness.  The  set  of  points  I  =  {o,  1,2,3,...} 
of  is  not  a  compact  set,  because  the  sequence  {°> 1,2,3,...}  c  I  doe 8  not  contain 
any  convergent  subsequence.  Therefore,  the  Euclidean  space  R1  (and  likewise  Rn) 
is  not  a  compact  metric  space.  One  can  show  that  every  subset  of  points  on  the 
real  line  R"*",  or  of  the  Euclidean  space  Rn  is  a  compact  set  if  and  only  if  it  is 
closed  and  bounded.  (For  instance,  the  set  a  <  x  <  b  is  a  compact  set.  ) 

Example  1 

Show  that  the  set  of  all  continuous  real-valued  functions  defined  on  the 
real  interval  [a,b]  with  distance  function 

d(f,fi)  =  max  ^|f(t)  -  g(t)|^  ,  a  <  t  <  b 

forms  a  separable  metric  space. 

Solution:  It  is  readily  seen  that  the  suggested  distance  function 

satisfies  the  requirements  of  axioms  1  to  4.  To  see  whether  a  metric  space  is 
a  separable  space,  consider  all  polynomials  with  rational  coefficients.  There 
are  countably  many  such  polynomials.  By  virtue  of  the  Weierstrass  approximation 
theorem,  any  element  of  the  space  can  be  uniformly  approximated  by  a  polynomial 
with  rational  coefficients.  Therefore,  this  is  a  separable  metric  space. 

1-3 .  Normed  Linear  and  Banach  Spaces 

In  Chapter  k  of  TR-65-399  we  have  defined  metric  spaces  of  finite  dimen¬ 
sions.  In  this  chapter  the  same  basic  concepts  are  presented  in  a  more  general 
fashion,  and  without  regard  to  finiteness  or  infiniteness  of  dimensions.  Consider 
a  linear  vector  space  V  with  finite  or  infinite  dimensions.  A  norm  in  a  linear 
vector  space  is  defined  as  a  real  number  associated  with  every  element  x  of  the 
space  (denoted  by  ||jc||)  having  the  following  properties. 
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1. 

Ilxll  > 

0 

for 

every  x  e  V 

2. 

fell  = 

0 

if 

and  only  if  x  =  0 

3- 

l|\x|i  = 

=  M  ■  fell 

for 

any  j;  e  V  and  any  scalar  h 

k. 

fettll 

<  fell  +  fell 

for 

any  pair  of  elements  in  V. 

The  relation  between  normed  linear  spaces  and  metric  spaces  is  a  simple  one. 

Let  fell  be  the  norm  of  an  element  jc  of  i  linear  vector  space  V;  define  the 
following  distance  function  over  the  elements  of  that  space. 

d(x,y)  =  fe-icll  =  fe-xl!  (1-13) 

The  fact  that  this  is  a  permissible  distance  function  is  an  immediate  consequence 
of  the  aforementioned  properties.  In  fact,  (1),  (2),  and  (3)  can  be  easily- 
verified.  To  show  the  validity  of  (4),  note  that 

d(x,y)  =  fe-xll  =  ll(x-z)  +  (z-y)ll  (1-14 ) 

<  fe-z||  +  llz-yll  =  d(x,  z )  +  d(z,y) 

Thus  the  normed  linear  spaces  are  metric  spaces. 

In  view  of  the  material  of  Section  1-2,  we  are  now  in  a  position  to  study 
metric  spaces  where  a  concept  of  convergence  of  vectors  has  been  introduced.  We 
say  that  £  is  the  limit  of  a  sequence  xn  ,  i.e., 

x  =  lim  xn  (or  xn->£)  if  fen-x||  -»  0  (1-15) 

n  -»  00  n  — > 

This  type  of  convergence  is  convergence  in  the  norm.  If  every  fundamental  sequence 
of  a  linear  normed  space  has  a  limit  in  that  space,  then  the  space  is  said  to  be 
a  complete  linear  normed  space  ■  A  linear  normed  space  which  is  complete  with 
respect  to  its  norm  is  also  referred  to  as  a  Banach  space  (named  after  the  Polish 
mathematician  S.  Banach).  Our  plans  do  not  require  a  detailed  mathematical  study 
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of  normed  spaces  and  Banach  spaces.  Nonetheless,  we  note  in  passing  that  many 
familiar  properties  of  ordinary  Euclidean  spaces  are  valid  for  this  broader 
class  of  Banach  spaces.  The  key  to  the  generalization  lies  in  the  fact  that 
Banach  spaces,  like  ordinary  real  Euclidean  lines,  are  linear,  normed  and  com¬ 
plete.  Some  of  the  familiar  concepts  which  may  be  directly  extended  from  the 
ordinary  Rn  to  Banach  spaces  are:  the  concepts  of  linear  dependence  of  elements 
linear  manifold,  subspace,  plane  and  sphere,  and  convergence  in  norm.  The 
concept  of  orthogonality,  the  elegant  properties  of  projection,  and  the  least 
square  distance  criterion  are  not  inadvertently  maintainable  in  a  Banach  space. 
The  applicability  of  these  latter  concepts  is  restricted  to  the  Hilbert  space 
which  is  a  subclass  of  the  Banach  spaces.  This  matter  will  be  discussed  shortly 

Example  2 

Show  that  the  space  Rn  is  a  Banach  space. 

Solution:  Rn  is,  of  course,  a  linear  normed  space.  It  remains  to  show 
that  it  is  complete  with  respect  to  the  Euclidean  norm.  Let 


i*iW*l'*2'*  ••  }  '  *k  =fxlk’x2k'-  -'xnkj 
be  a  fundamental  sequence  of  elements  in  this  space;  that  is,  for  every  e  >  0 
there  exist  an  N£  such  that  for  p,  g  >  N£  we  have 


d(x  ,x  )  = 
V~P  ~g 


-J./2 

Kp-V2]  <e 


2 

If  the  sum  of  a  finite  number  of  non  negative  terms  is  smaller  than  €  then 

2 

everyone  of  these  terms  must  be  smaller  than  €  }  hence. 


|xkP  -  ^gl  < € 


k  =  1,2,  . .  .,n 
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That  is 


l  xkl'xH2'"'  }  is  a  fundamental  sequence  of  real  numbers. 


Let 


=  lim  \v 

K  p  _♦  oo  Kp 


k  =  1,2, . . ,,n 


and 

S  =  I2,  Sn} 

Then  it  becomes  clear  that 


lim  =  25 
n  ->  oo 

Example  3 

Consider  the  half-open  set  of  real  numbers  between  zero  and  one, 
including  one  but  excluding  zero.  Show  that  in  this  space  the  sequence  xn  =  i 
is  a  Cauchy  sequence  which  is  not  converging. 

Solution:  The  above  Cauchy  sequence  has  no  limit  point  in  the  specified 

space.  However,  if  the  apace  was  to  include  zero,  then  the  Cauchy  sequence  would 
converge  to  a  point  in  the  space.  The  latter  space,  ^  x:  0  <  x  <  1^  ,  is  a 
complete  metric  space.  This  is  an  example  of  a  metric  space  which  can  be  made 
complete  by  adjoining  additional  elements. 

Example  U 

a)  Show  that  the  set  of  rational  numbers  form  a  metric  space  with  respect 
to  the  norm 


where  x  is  a  rational  number. 

b)  Is  this  a  Banach  space? 
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Solution:  Fart  (a)  is  straight  forward.  In  order  to  answer  (b),  take, 
for  instance,  the  sequence: 


w-a*.  I  •••) 

This  is  a  fundamental  sequence  since  it  converges  to  zero  which  is  an 
element  of  the  space.  On  the  other  hand,  the  set 

£rml  -  fu  +  j)1  >  a  + 1)2 ,  (1  +  j)3,  •••} 

has  no  limit  in  the  space  of  rational  numbers,  since 

lim  (l  +  i)m  =.  e 
n  -*  °°  m 

This  is  a  simple  example  of  a  metric  space  in  which  there  are  fundamental 
sequences  which  do  not  converge  to  a  limit  in  that  space.  This  is  not  a 
B- space. 

Example  5 

The  space  of  continuous  function  C[a,b]  is  a  Banach  space.  The  distance 
between  vectors  x(t)  and  y(t)  is  assumed  to  be  the  absolute  value  of  their 
largest  deviation  in  the  interval  [a,b]. 

d-USiX)  =*  Max  |x(t)  -  y(t)[  ,  a  <  t  <  b 
Solution:  Consider  a  fundamental  sequence 

Xg(t), 

Thus  for  a  given  e  >  0  there  exists  an  such  that  for  n,m  >  N£ 

K  -  *n»  <  e 
or 

|xn(t)  -  xn(t)|  <  €  a  <  t  <  b 
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This  implies  that  the  sequence  converges  uniformly  and  its  limit  is  a  con¬ 
tinuous  function  x(t )  such  that 


|xn(t)  -  X(t)|  <  e 


n  >  N 
a  <  t  <  b 


This  implies  the  convergence  in  the  sense  of  the  applied  metric. 

Example  6 

Show  that  the  space  of  square  integrable  continuous  function  (?  [a,b  ]  is 


not  a  Banach  space.  The  distance  being  defined  by: 


d(x,y)  =  (J  [x(t)  -  y(t)]2  dt 


1/2 


Solution:  It  is  not  difficult  to  show  that  the  space  of  all  continuous  functions 

<?[  a,b]  is  a  metric  space. 

In  order  to  show  that  this  metric  space  is  not  a  B- space,  let  us  simply 
construct  a  Cauchy  sequence  in  the  space  ^ [~1» 1]  which  does  not  converge  to  a 
vector  in  the  same  space. 

Let 

fn(t)  =  0 


f  (t)  =  nt 
n'  ' 


for  -1  <  t  <  0 
for  -O  <  t  <  l/n 
fn(t )  =  1  for  l/n  <  t  <  1 

The  sequence  £fn(t  is  indeed  a  sequence  of  continuous  functions.  But  as 
n  approaches  infinity,  we  find; 

lim  f  (t)  =  0  for  -1  <  t  <  0 

n  -»  oo  n 


lim  fQ(t)  =  1 


for  O  <  t  <  1 
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The  limit  vector  Is  a  discontinuous  function.  You  may  wish  to  show  as  an 
exercise  that  the  sequence 

£gn(t)J  =  arc  tan  nt  for  -1  <  t  <  1 

is  also  a  fundamental  sequence  which  converges  to  the  discontinuous  function: 


-  -*/2 

for  t  <  0 

gto(t)  =■  n/2 

for  t  >  0 

l-1*.  Abstract  Hilbert  Spaces 

As  far  as  the  application  of  the  concepts  of  abstract  space  to  physical 
sciences  is  concerned,  the  so-called  Hilbert  space,  commonly  denoted  by  H, 
occupies  the  dominant  place.  Hilbert  space,  or  Hilbert  function- space,  is  a 
space  of  finite  or  infinite  dimensions  defined  over  the  field  of  complex  numbers 
having  the  following  main  characters .  There  should  be  a  suitable  distance 
function  defined,  that  is,  the  metric  should  m  '  3e  from  an  inner  product  form. 
The  space  must  be  complete,  i.e.,  it  should  possess  the  convergence  property 
for  all  its  fundamental  sequences  with  respect  to  its  metric.  Furthermore,  the 
elements  of  the  apace  must  have  a  certain  property  of  "closeness"  which  is 
referred  to  as  the  separability  requirement. 

Hilbert  space  as  defined  here  is  an  inner  product  space  which  is  complete 
with  respect  to  its  norm,  and  may  be  separable  or  non  separable.  More  specifi¬ 
cally,  the  following  axioms,  1,  2,  3,  and  4  are  required: 

1.  H  is  a  linear  vector  space  over  the  field  of  complex  numbers. 

With  every  pair  of  elements,  x,y  of  there  is  associated  a 
complex  number  (x,y)  called  the  scalar  or  inner  product  of  jc 
and  y,  with  the  properties : 


2. 


“)  (as,x)  =  (¥>£) 

(Bar  denotes  the  complex  conjugates;  note  that  (x,y)  is 
real,  and  the  number  (x,x)^/2  =  ||g||  is  called  the  norm 
of  (x). 

b)  (x+z,i)  =  +  (z,x) 

c)  (Xx,y)  =  X(.jc,.y)  for  arbitrary  complex  number  X 

d)  (£,x)  >  0 

e)  (x,jc)  =  0  if  and  only  if  £  =  0 

3.  H  is  complete  with  respect  to  the  metric 

d(x,X)  =  lbc-y|| 

4.  For  every  positive  integer  n,  the  space  contains  n  linearly 
independent  element  s . 

5.  H  is  a  separable  space.  (This  property  is  optional  as  a  great 
part  of  the  theory  Hilbert  function- space  is  applicable  without 
regard  to  separability. ) 

We  note  in  passing  that  if  properties  (l)  and  (2)  are  satisfied,  the  space 
is  referred  to  as  a  Pre- Hilbert  space,  also  a  Unitary  space.  An  n-dimensional 
unitary  space  is  a  complex  Euclidean  space .  In  the  definition  of  Hilbert  spaces, 
it  is  not  unusual  to  forego  the  separability  requirement,  or  even  the  requirement 
of  infinite  dimensions,  at  times.  In  the  present  work,  however,  we  shall  always 
adhere  to  the  first  three  requirements,  but  will  not  insist  on  the  separability 
condition  for  every  problem  discussed.  A  Hilbert  space  which  satisfies  conditions 
(1),  (2),  (3),  and  (4),  but  not  (5),  is  referred  to  as  a  non  separable  Hilbert 
space.  Hilbert  space  is  a  subclass  of  B- space  which  arises  as  a  direct  generaliza- 
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tion  of  Euclidean  space.  In  view  of  the  fact  that  Hilbert  space  is  an  inner 
product  space,  its  geometry  is  closer  to  the  geometry  of  the  ordinary  Euclidean 
space  than  to  any  other  B- space. 


The  following  two  important  inequalities  for  elements  of  Hilbert 
are  direct  consequences  of  the  defined  axioms, 
a )  Schwarz's  inequality 

l(*,x)l  <  lUcll  llyll 

To  prove  this  inequality,  note  that  for  any  arbitrary  X. 
the  norm 

llg+Xy||  is  a  non  negative  number,  that  is 
(3  +  Xy,  £  +  >.y)  >  0 

Note  that  this  inequality  holds  trivially  if  y  =  0.  So  we 
assume  y  ^  0. 

_  g 

(x,x)  +  Mx,y)  +  X(y,jc)  +  |x|  (y,y )  >  o 

By  letting 

Os,x) 

X  = - 

One  obtains 

l(x^)  I2 

(jc,x)  -  -  >  0 

(y>x) 

or 

l(x,y)|  <  llisll  M 


spaces 

(1-16) 

(1-17) 

(1-18) 

(1-19) 

(1-20) 

(1-21) 


-16- 


b)  Triangle  inequality 


llx  +  y||  <  llxll  +  ||y II  (1-22) 

This  inequality  can  be  derived  in  a  similar  vein. 

II  (x  +  y)!!2  =  (x  +  y,  X  +  y)  (1-23) 

=  (x»  x)  +  (y,  x)  +  (x,  y)  +  (y,  y) 

<  llxll2  +  2  ||x||  |Jy||  -  liylf 

llx  +  y ||  <  ||x||  +  ||y||  (1-21*) 

2 

1-5.  Infinite-Dimensional  Euclidean  Space  -  Space  £ 

In  the  previous  section,  we  have  defined  the  basic  requirements  for 

abstract  Hilbert  spaces.  In  this  section,  we  propose  to  investigate  in  some 

detail  the  infinite-dimensional  Euclidean  space  as  an  example  of  Hilbert  space. 

2 

This  space  is  often  referred  to  as  the  space  £  .  The  coordinates  of  every  vector 
may  be  real  or  complex  numbers;  the  usual  vector  operations  and  the  norm  of  an 
element  are  defined  as  follows : 

x  =  (l^  l2,  ••.,  Sn,  ...  ) 

x  =  (nx>  v  •••■>  V  * 
x  +  X  =  (?!  +  V  *2  +  V  5n  +  V 

Xjc  =  (*-5^j  *-£n,  *  *  *  )  »  ^  a  complex  number 

The  inner  product  and  the  norm  are  respectively: 

OO 

(x,y)  =  (1-26) 

i=l 
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(1-27) 


+ 


00  for  any  $  e  £ 


The  condition  of  finiteness  of  length  is  essential  for  convergence 

2 

requirement,  and  must  be  included  in  the  definition  of  the  space  l  .  The 
first  thing  is  to  show  that  this  space  is  actually  a  linear  normed  (inner 
product)  space.  The  first  three  conditions  of  normed  spaces  are  obviously 
met.  The  completeness  of  this  space  can  be  inferred  from  Example  2,  for  a  real 
Euclidean  space  of  infinite  dimensions.  The  same  line  of  reasoning  is  essentially 

valid  when  the  components  are  complex  numbers.  It  can  be  also  shown  that  the 

2 

space  £  is  a  separable  space,  but  this  fact  is  not  of  an  immediate  concern  to 
our  studies. 

2 

The  following  two  basic  inequalities,  of  course,  are  valid  for  £  . 


■(£-■)  (S 


(1-28) 


These  inequalities  are  rather  interesting  and  often  lend  themselves  to  useful 
physical  interpretation  in  applications.  For  this  reason,  a  slightly  different 
derivation  is  presented  below.  In  order  to  show  directly  the  validity  of  the 
inequality 

Itell  <  fell  +  fell 

one  may  use  a  simple  and  well-known  variational  procedure.  Let  us  first  prove 
the  validity  of  these  inequalities  for  En  —  an  extension  to  the  case  of  ^  will 
present  no  difficulty.  Consider  the  finite  sets  of  real  numbers 
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[&x,a2,  ■  ■  -,an]  and  £b,,b2,  . . -,bn^  - 

A  familiar  variational  method  suggests  the  calculation  of  the  quadratic 
function 

n 

f(x)  =  (a^x  +  b^  )^  =  Ax^  +  2Bx  +  C  (1-30) 

i=l 

where  x  is  an  arbitrary  real  quantity,  and 


A=  £  (ai}  ’ 

n 

b  =  Y  aibi  > 

c.  £  <»*> 

(1-31) 

i=l 

i=i 

i=l 

Since  f(x)  must  remain  non  negative  for  all  real  values  of  x,  the  condition 


AC  <  O 


promptly  gives 


(1-32  ) 


a  .b . 
i  l 


U-i 


*(£■ ■•)  (£ 


(1-33 ) 


This  is  a  useful  inequality  which  was  originally  obtained  by  Cauchy.  Now  the 
triangle  inequality  can  be  derived  without  difficulty.  In  fact,  by  taking  the 
square  root  of  both  sides  of  Cauchy's  inequality,  multiplying  by  two,  and  adding 
A  +  C  we  find 

2  VT+  (A+C)  <  2  Va"  Yc"  +  (A+C)  (l-3l ) 


n 

£  (»i  +  Sajb.  +  b\)  <  (  •yfZxlc)2  (1-35) 

i=l 

or,  the  triangle  inequality: 
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-  n  -i  1/2 

|"  n  Tl/2  f  n 

"I1/2 

Z  (ai  +  bi)2  - 

.  i=l  J 

Z  ai2  +  z 

-i=l  J  Li=l 

(1-36) 

ihe  extension  of  Cauchy's  inequality  to  the  case  where  n  =  <*>,  (also  for  complex 
2 

£  spaces)  is  not  difficult.  The  only  restriction  is  that  A  and  C  should  remain 

finite;  this  is  guaranteed  by  virtue  of  the  assumption  of  finite  norm  for  all 

2 

elements  of  £  . 

2 

Separability  of  Space  £ 

2 

Consider  the  set  D  of  all  elements  of  £  having  only  a  finite  number  n  of 

non-zero  coordinates;  all  such  coordinates  being  rational  numbers.  We  will  show 

2 

that  D  is  dense  in  £  .  To  this  end,  for  any  x  and  e  >  0  we  set  forth  to  find  a 
suitable  converging  sequence,  by  means  of  a  point  z  e  D. 


x  « 

~  =  ^fl’  Z2  *  *  *  *  ’  ^’  °'  *  ”  ’^  ~  * 

lbs- s  II2  =  £  lvzk|2  +  Y.  !x*r 


x  e  £ 

D 


k=l 


Note  that  the  series 


I 


k=n+l 


is  convergent  by  hypothesis.  Therefore,  we 


k=n+l 


can  find  an  n  =  n  such  that 
o 


Y. 

k=n  +1 

o  n 

o 

Consequently,  in  the  finite  sum  ^  |x^-z^pwe  may  choose  the  rational  numbers 


k=l 


z^  close  enough  to  x^  such  that 
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n 

Z  |xk-zk|2  - 1 

k=l 


2 

e 


I!x-z||2  <  €2,  llx-z||  <  € 

2 

This  inequality  proves  that  D  is  dense  in  l  . 

Example  7 

2 

The  unit  sphere  of  the  infinite-dimensional  Euclidean  space  (Z  )  is  not  a 
compact  set. 

Solution:  To  show  the  validity  of  this  statement,  consider  for  instance  the 

infinite  zet  of  orthogonal  vectors  £j_  -  (l*  O,  . .  •  ),  ep  —  (0, 1, 0,  0,  •  •  •  ) 


Ik  I'  =  1 


1,  2 , 3 ,  . . . 


We  cannot  select  a  convergent  subsequence  from  since  the  distance  between 

any  pair  of  elements  is 


lie  “  sJ' 

n  ~m 


=  t§r 


e,e  -  e  )  =  2 
'-m  ~n  ~m 


1-6 .  The  Space  L  -  Function  Space 

An  important  realization  of  Hilbert  space  is  provided  by  the  so-called 

2 

L  space.  Let  [a,b]  denote  a  finite  or  infinite  interval  on  the  real  axis. 

.  * 

Consider  the  set  of  all  real  valued  square  integrable  functions  f (t ;,  i.e., 

b 


/ 


[f(t)p  dt  <  +  eo 


(1-37) 


Under  the  usual  definition  of  sum,  and  product  by  a  scalar,  and  defining  the 

zero  element  as  a  function  which  is  "equal"  to  zero  (almost  everywhere)  in  [a,b], 

we  have  a  linear  vector  space.  In  this  vector  space  induce  the  inner  product 
_  -  - 

f(t)  is  denoted  by  lower-case  letters  and  is  represented  in  the  vector  space  by 
a  vector  f. 
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(f,s)  = 


dt 


(1-38) 


b 

J  f(t)  g(t) 

a 

The  Integral  is  to  be  taken  in  the  sense  of  Riemann.  The  functions  space  thus 

2  2 
defined  is  referred  to  as  L  space.  It  will  be  shown  now  that  the  space  L  is 

a  linear  vector  space. 

2 

1.  L  is  a  linear  vector  space. 

2 

i)  The  product  of  any  two  elements  of  L  is  integrable,  since 


k(t)  g(t)|  <  i  [f(t)32  +  \  [g(t)f  (1-39) 

observation  is  needed  for  the  proof  of  (ii)  below. 

2  2 

ii  )  The  sum  of  any  two  elements  of  L  is  an  element  of  L  . 

Proof; 

[f(t)  +  g(t)]2  <  [f(t)]2  +  2  |f  (t )  g(t)|  +  [g(t)]2  (1-40) 

2  2 
iii)  If  f  £  L  and  X.  a  constant,  then  evidently  Xf  e  L  . 

J{_\  f(t)]2  dt  =  \2  dt  <  oo  (l-4l) 


2.  Properties  a,b,  and  c  for  the  suggested  inner  product  function  are 
easy  to  verify.  For  d  one  notes  that  the  integral  of  a  non  negative 
quantity.  Part  e  is  more  difficult  to  verify.  The  difficulty  arises 
on  account  of  some  mathematical  subtleties.  Take,  for  instance,  square 
integrable  functions  u(t)  whose  integrals  are  zero  but  the  functions  can 
tak<=  on  positive  values.  The  difficulty  can  be  circumvented  by  re¬ 
sorting  to  the  fact,  that  the  set  of  t  for  which  u(t)  >  0  has  the 
measure  zeroj  that  is,  a  set  of  points  on  [a,b]  which  can  be  covered 
by  a  finite,  or  countably  infinite,  set  of  intervals  with  an  arbitrarily 
small  total  length.  This  amounts  to  saying  that  any  non  negative  func- 
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tion  whose  integral  is  zero  differs  from  zero  element  on  a  set 


of  measure  zero.  One  may  say  that  such  an  element  is  equivalent 

2 

to  the  zero  element.  In  other  words,  the  null  element  of  L  is  the 
collection  of  all  functions  defined  on  the  real  axis  which  are  equiva¬ 
lent  to  zero.  Two  elements  differing  on  at  most  a  set  of  zero  measure 
are  referred  to  as  "equivalent".  The  integrals  of  the  square  of 
equivalent  elements  are  equal.  Therefore,  from  the  strict  mathe¬ 
matical  point  of  view  the  space  of  functions  considered  here  is 
rather  a  space  of  "equivalence  classes".  The  distance  between  two 
functions  f(t)  and  g(t)  equals  zero  precisely  when  f(t)  =  g(t)  for 
almost  all  t.  To  depict  this  situation,  let  us  write  f  '  g  if  said 
only  if  d(f,fi)  =  0.  We  do  not  pursue  this  matter  further j  a  presenta¬ 
tion  encompassing  measure  theoretic  consideration  (Theory  of  Lebesque 
integral)  is  not  within  the  scope  of  our  present  undertaking. 

2 

3.  The  L  space  is  a  complete  space. 

For  proof,  see  standard  texts,  such  as  W.  Rudin,  Real  and  Complex 
Analysis;  McGraw-Hill  Book  Co.j  Akhiezer  and  Glazman,  Theory  of 
Linear  Operators  in  Hilbert  Space,  Vol.  1,  pages  21-23  of  the  English 
translation. 

2 

4  /  5.  For  a  discussion  of  the  validity  of  these  properties  for  L  ,  see 

Chapter  1  of  the  last-cited  reference,  or  Chapter  VIII  of  Functional 
Analysis,  by  Kolmogorov  and  Fomin.  Detailed  proof  of  separability 
with  its  implications  can  be  found  in  Chapter  2  of  L.V.  Kantorovich 
and  A.P.  Akilov,  Functional  Analysis  in  Normed  Spaces,  Macmillan  Co., 
New  York,  1964 . 
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For  the  sake  of  exercise,  Schwarz'  inequality  and  the  triangle  inequalities 

2  2 
for  vectors  of  L  will  be  derived  directly.  Let  f  and  g  be  elements  of  L  not 

equivalent  to  the  zero  element,  and  X.  an  aribtrary  parameter. 


J  (*f+g f 

a 


dt 


dt  +  2\ 


u  u 

J  fg  dt  +  J  g2  dt 


(l-k2  ) 


This  quadratic  function  must  remain  non  negative  for  all  real  values  of  X., 
hence 


r  r  I2 

r  b  i 

^  _  i 

r  \  0  i 

/  fg  dt 

< 

« 

1/  g  dtj 

LJa  J 

-  a  J 

La  J 

(1-43) 


This  inequality  is  known  as  Cauchy- Bunyakovski ' s  inequality.  By  taking  square 
roots,  multiplying  by  two  and  adding  f  f?  dt  +  /  g"  dt  to  both  sides,  one 


LnequaxiLj 


obtains  an  inequality  bearing  the  name  of  Minkowski. 


U 

/ 


(f+gf  dt 


1/2 


(M 


1/2 


(1-44) 


These  inequalities  are  readily  recognized  as  the  Schwarz  and  the  triangle 
inequalities  discussed  earlier  for  Hilbert  spaces,  i.e.. 


I(f,g)l  <  Ilf  II  llgll  (1-45  ) 

llf+gll  <  Ilf II  +  llgll  (1-46) 

2 

In  the  definition  of  the  space  L  [a,b],  for  simplicity,  the  assertion  was 
made  that  these  functions  are  real  valued.  This  restriction  can  be  easily 
removed.  Let  <2>(t)  >  0  be  a  non  negative  square  integrable  function  on  (a,b]  and 
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denote  by  L^[a,b]  all  complex- valued  functions  square  integrable  with  respect 
to  the  weighting  function  0(t),  that  is: 


/ 


<Ht)  |x(t)|2 


dt  <  » 


(1-47) 


A  simple  extension  of  the  foregoing  material  will  enable  one  to  show  that 
2 

the  space  V«,b]  is  also  a  separable  Hilbert  space.  The  chosen  inner  product 
18  * 

(1-48) 


(z,x)  =  J  $(t)  [x(t)  y(t ) ]  dt 


Very  often,  in  application,  one  deals  with  the  case  0(t)  =  1;  that  is  L  [a,b] 

b 


=  J  x( t)  y(t)  dt 


(1-49) 


1-7.  Continuity  of  Scalar  Product 

We  wish  to  show  that  the  inner  product  is  a  continuous  function  with 
respect  to  the  norm  of  the  Hilbert  space.  Let  £  and  y  be  the  limits  of  the 
sequence  and  M  respectively.  By  virtue  of  the  preceding  inequalities 

one  can  write: 

l(xn>xn)  -  (as,x)l  S  Kasn>xn)  -  (a?n»x)l  +  lfen>x)  -  (x»jf)l  (--50) 

=  l(«n,xn-x)l  +  l(asn-x,i)l 

<  lli£nll  lbcn-xll  +  III  II  llasn-ali 

<  M||yn-y||  +  foil  ||xn-ac|| 
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where  M  denotes  the  upper  bound  of  Ila5nll  and  llynll.  As  n  is  infinitely  increased, 
we  find 

l|xn-x||  -+0  ,  ikn-xl!-*0  (1-51) 


n  oo 


n  — * 00 


whence 

l(xn,yn)  -  (x,y)  |  -*  0 


(1-52) 


This  proves  that  the  inner  product  in  a  Hilbert  space  is  a  continuous  function 
with  respect  to  the  norm. 


l-».  Linearly  Independent  Vectors 

The  reader  is  already  familiar  with  the  definition  of  a  finite  set  of 
linearly  independent  elements  of  a  finite  dimensional  metric  space. 

A  finite  set  of  elements  of  a  Hilbert  space  is  said  to 

be  linearly  dependent  if  there  exists  scalars  .  .  •  ,&n  ,  not  all  zero, 

such  that 

n 

£  “A  =  9  (1-53) 

i=l 


when  the  finite  set  is  not  linearly  dependent,  then  is  said  to  be  linearly  in¬ 
dependent.  The  following  statement  is  self-explanatory. 

Theorem:  The  necessary  and  sufficient  condition  for  a  finite  set  of  n 

vectors  of  an  innei  product  space  to  be  linearly  independent  is  that  its 
Gram  Determinant  be  different  from  zero,  i.e.: 


($^4) 

($n,$  ) 


<*l'*n> 

^2'V 

<*n'*n> 


/  0  (1-5^  ) 
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An  infinite  set  of  vectors  of  a  Hilbert  space  is  said  to  be 

linearly  independent  if  every  finite  subset  of  it  is  linearly  independent. 

It  is  natural  to  extend  the  concept  of  basis  from  a  finite-dimensional  metric 
space  to  a  Banach  space  of  infinite  dimension.  When  the  number  of  elements  in 
a  set  of  linearly  independent  elements  of  the  space  becomes  infinite,  the  space 
is  said  to  be  of  infinite  dimension.  This  number  may  be  countable  or  uncountable. 

In  this  elementary  exposition  we  only  consider  Hilbert  spaces  of  countable  dimension, 
or  simply  Euclidean  spaces  of  infinite  dimension. 

Let  fe^gg,  .  .  *,§n,  •  •  be  a  countable  set  of  independent  elements  of  a 
Banach  space  X  such  that  every  element  x  e  X  admits  a  unique  representation 


x  = 


k=l 


That  is,  to  each  x  there  corresponds  a  unique  sequence  x^>  *2>  *  •  •  >  xn» 
scalars  such  that 


lim 
m  -»  oo 


0 


(1-55) 


.  . .  of 


(1-56) 


Under  these  conditions  we  say  that  X  has  a  countable  basis.  In  our  study  we 
simply  adhere  to  the  case  where  the  space  has  a  countt  jle  basis.  A  discussion 
of  B-spaces  with  uncountable  basis  is  indeed  beyond  our  plans.  It  can  be  proved 
that  any  Banach  space  with  a  countable  basis  is  also  a  separable  space.  Also,  we 
are  told  that  the  following  question  is  an  unsolved  problem  of  modern  mathematics. 
"Determine  whether  for  every  separable  Banach  space  there  exists  a 
countable  basis." 

See,  for  instance,  A.E.  Taylor,  General  Theory  of  Functions  and  Integration, 
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Blaisdale  Publishing  Co.,  New  York,  1965,  page  15<?«  The  set  of  all  linear  com¬ 
binations  of  subsets  of  an  infinite  set  of  independent  elements  is  referred 

to  as  the  linear  subspace  generated  by  [ (also  linear  hull). 


1-9-  Linear  Manifold  and  SubBpace 

A  non  empty  subset  of  elements  of  H  is  said  to  form  a  linear  manifold  L 
if  for  tiny  .£  e  L,  g  €  L  and  arbitrary  numbers  a  and  3  we  have 


af  +  p  Q  e  L  (1-57) 

A  subset  of  elements  in  a  linear  space  is  said  to  be  closed  if  the  set  contains 
all  its  limit  points.  In  view  of  this  definition,  we  note  that  in  contrast  with 
the  finite-dimensional  inner  product  space,  a  linear  manifold  in  a  Hilbert  space 
may  not.  be  necessarily  closed.  A  closed  subspace  is  itself  a  Hilbert  space.  Trivial 
examples  of  closed  subspaces  are  the  whole  space  H,  and  the  null  space  of  H  con¬ 
taining  the  zero  element  only. 

As  an  example  of  a  linear  manifold  which  is  also  a  closed  subspace,  consider 

2 

the  following  type  of  vectors  of  l  : 

t  =  '  fl  “  f2  *  f3  (1-58) 

&  =  9  =  £*2  =  6^  (l— 59 ) 

Evidently  all  points  x  =  Clf  +  3g  will  have  coordinates  of  the  form 


X1,X2,X3,3VX5  >•••}  > 


X1  =  *2  =  *5 


(1-60) 


The  points  of  H  satisfying  x^  =  x^  =  x^  form  a  linear  manifold  L  which  is  also 
a  closed  subspace. 

As  an  example  of  a  linear  manifold  which  is  not  a  closed  subspace,  consider 

2 

the  set  L  of  all  vectors  in  l  which  have  a  finite  number  of  non  zero  coordinates 

relative  to  some  basis.  Then  the  sequence  of  vectors 
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Xg  > 


where 


-  £l,  0,  0,  0,  .  .  .J 

&2  z  ^->2> 

Sj  =  °>  • '  '  j 


is  a  converging  sequence  of  vectors  in  L.  However,  the  limit  vector  of  this 
sequence  does  not  belong  to  L. 


lim  x 
n  -♦ 


1  1  1 


1-10.  Orthogonality 

The  elements  x  and  y  of  H  are  said  to  be  orthogonal  if 

(as,*)  =  0  (1-61) 

Orthogonal  elements  are  denoted  as  x  J_  y.  An  element  x  e  H  is  said  to  be 
orthogonal  to  a  subspace  S  of  H  if  x  is  orthogonal  to  every  element  of  S,  and 
we  write  x  Is.  If  elements  of  two  sets  S1  and  Sg  are  pairwise  orthogonal,  the 
sets  will  be  referred  to  as  orthogonal:  1  Sg. 

The  sets  of  all  elements  orthogonal  to  a  given  set  S  is  a  subspace  of  H. 

This  subspace  is  called  the  orthogonal  complement  of  S.  It  is  not  difficult  to 
see  that  if  x  J_  y  and  x  J_  z,  then  x  ]_  a^y  +  a^z.  Also  in  view  of  the  continuity 
of  the  scalar  product,  we  find  that  if  x  ]_  y^  (n=l,2,  ...)  and  y^  -» y,  then  x  y. 

The  definition  of  length  of  a  vector,  and  the  angle  between  vectors  in 
Hilbert  space  are  identical  with  these  definitions  in  finite  dimensional  Euclidean 
space,  i.e  , 

lias  ||  =  +  ’V(x,x) 

(as,*) 


lias  II  =  +  V(as,as) 

cosine  of  the  angle  between  x  and  y  = 


fell  fell 
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A  set  of  elements  e^,.g2>  •  •  •  >£n  of  a  Hilbert  space  is  said  to  be  orthonormal 


if 


(e  e.  ) 


1 

0 


for  i  =  j  ) 


for  i  ^  j 


(1-62) 


V  I 

It  is  easy  to  show  that  every  orthonormal  set  of  vectors  is  an  independent  set. 

A  set  of  orthogonal  or  orthonormal  elements  of  H  is  said  to 

be  complete  if  there  does  not  exist  a  non  zero  vector  in  H  orthogonal  to  each 
vector  of  the  set.  Problems  concerning  the  completeness  of  a  set  of  orthogonal 
vectors  in  a  metric  space  are  rather  complex.  Here  we  quote  some  of  the  results 
without  proof. 

1.  There  is  a  complete  orthonormal  set  in  any  non  empty  inner-product 
space. 

2.  If  H  is  a  separable  space  then  there  exists  in  H  a  complete  ortho- 
nornal  set  with  at  most  denumerable  elements. 

Every  finite  or  infinite  set  ^h-^lv,,  . .  .,1$^,  .  .  .^  of  linearly  independent  elements 
of  H  can  be  used  to  obtain  a  set  of  orthonormal  elements  by  an  orthogonalization 
process.  This  process,  in  essence,  was  described  in  an  earlier  chapter  for  inner 
produce  spaces  with  finite  dimensions. 

In  Section  1-13  we  shall  present  a  procedure  for  construction  of  a  set  of 
orthonormal  vectors  in  a  Hilbert  space.  A  Hilbert  space  may  contain  a  countable, 
or  an  uncountable  number  of  such  orthonormal  vectors.  In  the  first  case,  the  space 
is  a  Hilbert  space  of  countabi e  uimension.  Finite- dimensional  spaces  and  spaces 
of  countable  dimensions  can  be  also  defined  as  separable  spaces. 

Example  1-8 

Let  h  be  an  arbitrary  point  of  H  except  0;  show  that  the  set  of  aid. 
points  of  H  orthogonal  to  h  forms  a  subspace  of  H. 

Solution:  In  fact,  if  f  ]  h  and  q  J_  h,  then  due  to  the  bilinear  property 
of  the  inner  product  function,  we  find  that  (af  +  0g)]_  h. 
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1-11  Isomorphism  of  Separable  Hilbert  Spaces 

The  definition  of  isomorphism  between  two  separable  Hilbert  spaces  can  be 
formulated  as  a  generalization  of  the  definition  of  isomorphism  between  finite 
dimensional  Euclidean  spaces.  Two  separable  Hilbert  spaces  H  and  H'  are  said 
to  be  isomorphic  if  there  exists  a  one-to-one  correspondence  between  their 
elements  such  that: 

1.  If  x'  e  H'  corresponds  to  k  e  H,  and  x'  e  H'  corresponds  to  y  e  H, 
then 

x'  +  y'  €  H'  should  correspond  to  x  +  x  e  H. 

2 .  For  any  arbitrary  number  \  of  the  field, 

e  H'  should  correspond  to  \x  e  H. 


Moreover,  the  two  spaces  are  said  to  be  isometric  if 
5-  d(x,y)  =  d(x '  ,y ' ) 

It  is  not  difficult  to  show  that  all  Hilbert  spaces  of  countable  dimensions 

2  2 

are  isomorphic  to  each  other.  In  particular,  the  spaces  L  [a,b]  and  l  are 
isomorphic  and  isometric. 

Let  ^e^,^,  . .  .^  be  a  complete  orthonormal  system  in  a  separable  Hilbert 
space  H,  and  x  an  arbitrary  element  of  H.  Then  x  admits  the  (Fourier)  repre¬ 
sentation:  „ 

OO 

*  “  Z  ^ « k 

k=l 


where 


OO 

2 

xk  =  ^x’ek^  and  f*k  1  <  " 

k=l 
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2 

We  can  establish  a  one-to-one  correspondence  between  points  of  H  and  l  , 

2 

for  instance  by  selecting  x'  e  l  to  correspond  to  x  e  H  such  that  x'  has  the 
same  coordinates  as  x,  i.e.. 


X  f= 

^x1,x2,...} 

relative 

tQ  £s1's2'  * 

X  1  = 

fxl’  *2»  •  *  *1 

relative 

to  £g{>  e^,  . 

2 

This  correspondence  obviously  establishes  an  isomorphism  between  H  and  Z  .  In 

order  to  see  whether  the  two  spaces  are  isometric,  let  us  calculate  the  inner 

product  function  for  two  arbitrary  elements  x  and  y  of  H,  and  their  corresponding 
2 

images  x*  and  y '  of  l  .  With  seme  mathematical  care  about  the  convergence  of  the 
incurring  sequences,  we  find: 


(x',y') 


Yk 


b  b  I  °°  \  I  00  \  00 

(*»*)  =  /  x(t)  ^(t)  dt  =  /  (  Z  Z  dt  =  Z  Yk 

a  a  »  k=l  *  V  k=l  /  k=l 

2 

Thus,  any  separable  Hilbert  space  is  isometric  with  Z  . 

The  separability  argument  enters  the  discussion  in  view  of  the  necessity 
of  representing  every  vector  of  the  space  by  its  coordinates  (the  so-called  Fourier 
coefficients).  (See,  for  example,  Liusternik- Sobolev,  page  79  of  English  trans¬ 
lation,  Ungar  Publishing  Co.  ) 


1-12  Projection  of  a  Point  on  a  Subspace 

In  many  problems  concerned  with  application,  one  wishes  to  find  the 
"shortest"  distance  between  a  given  point  x  of  a  Hilbert  space  and  a  subspace  S 
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o t  that  space.  An  intuitive,  but  rather  illuminating  way  of  looking  into  this 
problem  is  to  search  for  the  point  y  which  may  be  intuitively  referred  to  as 
the  "projection"  of  x  on  S.  This  is  schematically  illustrated  in  the  figure 
below.  In  a  rigorous  mathematical  approach,  one  has  to  show  that  there  exists 
a  unique  point  y  e  S  which  could  justify  the  definition  of  projection.  To 
fulfill  this,  we  require  (a)  that  x  -  y  be  orthogonal  to  the  subspace  S;  and  (b  ) 
that  y  be  the  "nearest"  point  of  S  to  x,  in  symbols: 

(a)  x  -  y  [y1  y*  e  S  (1-6.5  ) 

(x  -  y,  y' )  =  0 

(b)  lix-yll  =  inf  ||x-y'|l  <  llx-y'll  (l-CU) 

y 'eS 


r~x 


Fig.  1-5-  Projection  on  a  Subspace 

We  prove  formally: 

Projection  Theorem.  Given  x  €  H  and  S  a  complete  subspace  of  H,  there 
exists  a  unique  point  y  e  S  satisfying  requirements  a  and  b  above. 

Since  |]x-y'||,  (for  all  y'  e  S)  is  bounded  below  by  0,  it  has  a  greatest  lower 
bound,  inf  llx-y'll  =  d  >  0.  Hence  there  exist  a  sequence  dR  =  llx-y^ll^  which 
converges  to  d.  Let  i*nl  be  a  sequence  in  S  such  that  d^  =  ||x-ynlP  approaches 
d  as  n  -» =».  Furthermore,  let  h  be  an  arbitrary  element  of  S,  and  \  an  arbitrary 
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complex  number.  Of  course,  yn  +  \h  is  an  element  of  S  and  by  hypothesis 

I  lit  -  (yn  +  Xh)!!2  >  d  (1-66  ) 

Since  the  space  possesses  an  inner  product,  we  may  write: 


llx-yjf2  "  ^(J?,x-yrn)  -  ^(x-in>h)  +  |\  f  llhll2  >  d  (1-67 ) 
This  inequality  must  hold  for  any  desired  value  of  In  particular,  letting 


\ 


(x-2n>h) 

ifeiF 


yields 


l(x-yn,h)  |2 

teiP 


>  d 


(1-68) 


(1-69) 


(dn-d)  llhll2  >  l(x-yn,h)|2 


(1-70) 


iy  VdTTd  >  |(x-xn,h)|  (1-71) 

Thus,  for  any  yn  and  y^  of  the  S  we  have 

l(xn-i^})j)l  <  I  (Yn-as,k)l  +  f(iS-Xm/l})l  < 

_ _  , _  (1-72 ) 

( -y  dn-d  +  Vd^i)  ||h|| 

For  h  =  y^  -  y^,  we  find 

|kn-\ll<V3?'  +  (1-75) 

This  inequality  leads  to  the  conclusion  that  the  sequence  •  •  •  >Xn>  •  •  • 

is  a  fundamental  sequence,  and  converges  to  a  point  y.  We  will  now 
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show  that  y  e  S  is  in  fact  the  desired  projection  point  defined  by  d  =  ||x-y|j 

=  inf  llx-y'll  as  follows: 

J'e  S 

l|x-yll  <  Ils-yJI  +  lly-ynll  (1-7*0 

Taking  the  limit  of  both  sides  yields: 

llx-y||  <  d  +  0  (1-75) 

but  by  hypothesis  ||x-y||  >  d  for  all  y  £  S,  hence 

lix-yll  =  d  (1-76) 

Taking  the  limit  in  inequality  1-71  yields: 

(x-y>h)  =  o 

*-X  1*5 

Since  h  is  an  arbitrary  element  of  S,  it  follows  that  jc-y  J_  S. 

In  order  to  show  the  uniqueness  of  this  element  y,  let  g-y  =  z  and  assume  that 
y'  /  y  is  another  point  of  S  with  the  desired  property.  We  have 

x  =  X  +  z  X  e  S  ,  z  is  (1-77) 

x  =  y'  +  z'  y'  e  S  ,  z'  ]  8  (1-78) 

These  relations  suggest  that 

X  -  X*  e  S  ,  z-z'is  (1-79) 

whence 

Hy-y’ll2  =  (y-y'.,y-y')  =  (z'-s,y-y’ )  =  o  (i-8o) 

The  latter  equality  implies  that  y  and  y_  are  coincident.  This  unique  point  y 

with  the  described  property  is  the  projection  of  x  on  the  subspace  S. 

- 

An  alternative  proof  may  be  based  on  the  continuity  of  the  inner  product,  that 
is: 

d  =  lim  d  =  lim  l|x-y_ll  =11  x  -  lim  y  II  =  ||x-y|| 
n-»°°  n  — * 00  n  — ► 00  u 
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In  particular,  if  the  subspace  S  is  the  linear  manifold  generated  by  a  sole 
vector  g,  then  the  projection  of  x  on  that  manifold  is  the  vector  y  such 
that 

=  0  (1-81) 
Now  y  can  8e  expressed  as  y  =  \g,  whence 


(*,  &) 

\  =  - 

(e,  &) 


Fig.  1-7 

Projection  on  a  multi¬ 
dimensional  subspace 


(1-82) 


Fig.  1-6 

Projection  on  a  one- 
dimensional  subspace 


The  projection  of  x  on  g  is: 


Froj .  g  on  g  = 


(x,  g) 

(s>  s) 


& 


(1-85 ) 


Moreover,  when  g  is  a  vector  of  unit  length,  one  arrives  at  the  simplified 
familiar  relation 

Proj.  x  in  g  =  (x,g)  g  (1-84) 
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If  a  subspace  S  is  generated  by  the  linear  manifold 


Z  Vk' 


where  e, 
~k 


k=l 


is  an  orthonormal  set,  it  is  not  hard  to  arrive  at  the  simpler  result 

n 


Proj.  jc  on  S  =  (x,gk) 


(1-85) 


k=l 


1-13  A  Procedure  of  Orthonormaiization 

Two  sets  of  vectors  in  a  Hilbert  space  are  said  to  be  equivalent  if  and 
only  if  each  element  of  one  set  is  a  finite  linear  combination  of  the  elements 
of  the  other  set.  If  every  pair  of  vectors  in  a  set  of  elements  of  a  Hilbert 
space  Eire  mutually  orthogonal  vectors,  then  the  set  is  said  to  be  an  orthogonal 
set.  Moreover,  if  the  elements  of  an  orthogonal  set  have  unit  length,  the  set 
will  be  referred  to  as  an  orthonormal  set. 

In  the  sequel  a  method  is  presented  for  constructing  an  equivalent  ortho¬ 
normal  set,  £e^,eg,  . .  .,gn,  . .  .J  for  a  given  finite  or  infinite  set  of  independent 
vectors  •  •  ■  j  •  I** 

ei  =  HS^II  (1-86) 

Denote  by  the  one-dimensional  space  generated  by  either  vector  e^  or  g^. 

Next,  find  the  projection  of  gg  on  space  E^,  and  calculate  the  non  zero  vectoir 
hg  orthogonal  to  E^. 

^2  =  %  ‘  §!  4  0  (3.-87) 

Thus  hg  and  are  orthogonal  pairs,  and  hg  /  0  due  to  linear  independence  of 
g^  and  gg.  In  order  to  construct  eu.  orthonormal  pair,  ^eg,  let  simply 
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(1-00) 


jjg 

%  =  usp 

Now  and  eg  are  orthonormal  basis  of  a  two-space  Eg.  We  proceed  likewise 
by  finding  the  projection  of  g^  on  Eg,  and  by  continuing  the  process  described 
earlier  we  find  a  vector  h^  /  0  orthogonal  to  Eg . 


Normalization  yields 


(1-89) 


(1-90) 


The  vector  en,  for  any  positive  integer,  can  be  constructed  in  e  similar  fashion 


n-1 


®n  =  gn  -  Y.  (V%)ek  * 


(1-91) 


k=l 


(1-92) 


A  set  of  orthonormal  vectors  of  H  is  said  to  be  a  complete  orthonormal  system, 

if  the  system  is  maximal,  that  is,  there  exists  no  other  element  different  from 

2 

0  and  orthogonal  to  every  Hilbert  space.  For  example,  for  L  [-jr, n]  by  using 
vectors  of  the  form  sin  nt,  cos  nt,  we  can  derive  the  following  complete  ortho¬ 
normal  system: 

1  cos  t  sin  t  cos  2t  sin  2t  cos  nt  sin  nt 

yr; '  yv  >  y-p  ’  yr  '  y—  '  *•*'  vr  '  yr  '  " 

(1-93) 

It  is  impossible  to  add  any  other  orthonormal  non- identic  ally  zero  element  to 
this  system. 
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1-l^t  Fourier  Representation  in  a  Hilbert  Space 


As  discussed  before,  the  problem  of  representation  is  to  ascertain  the 
possibility  of  characterizing  every  element  of  the  space  as  a  linear  combination 
of  the  elements  of  a  given  basis  of  that  space.  In  other  words,  to  represent 
a  vector  by  its  so-called  "coordinates"  with  respect  to  a  certain  basis. 
Accordingly,  in  this  section  we  study  the  representation  of  any  vector  h  e  H 
with  respect  to  a  complete  orthonormal  set  of  vectors 

C'i  =  [<“-!»%>  •••»£„'*  “3  • 

It  will  be  shown  that  the  problem  of  determination  of  the  coordinates 
of  h,  and  the  problem  of  least  square  approximation  of  h  by  a  linear  combination 
of  elements  of  e  are  essentially  the  same.  Both  these  problems  in  turn  coincide 
with  the  finding  of  the  projection  of  h  on  the  subspace  specified  by  the  ortho¬ 
normal  set. 

The  problem  is  to  search  for  the  "best"  linear  expansion  of  the  vector  h 
in  terms  of  the  elements  of  the  specified  orthonormal  set 

00 

fe  ~  51  ai  ~i  (1-9^ 

i=l 

What  are  the  "best"  coordinates  £ai> a2, • • • , an,  • • >J  for  representation  of  h? 

By  the  "best"  it  is  meant  that  no  other  set  of  ®2» *  * • » anl,  should  lead  to 

a  lower  error  in  norm,  i.e.,  an  error  smaller  than: 

OO 

life  -  51  ai  (1-95) 

i=l 

This  is  the  problem  of  least  square  approximation,  or  Fourier  series  expansion 
in  Hilbert  space. 
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We  begin  by  studying  the  problem  of  the  least  square  approximation  in 
spaces  of  finite  dimensions. 

Theorem 

Let  be  a  vector  of  a  finite-dimensional  inner  product  space  with  an 
orthonormal  set  of  vectors  *  the  m^n^nam  ° ^  the 

expression 

lib  -  ^  ai  Sill  (1-96) 

i=l 


over  all  possible  values  of  a^'s  corresponds  to  the  selection  of 


ai  =  (b»e^)  i=l,2,...,n 


(1-97) 


For  any  other  values  of  a^s  we  will  have 


lib  -  £  (6,%)^  II  <  ilh  -  £  a^ej 


(1-90) 


i=l 


i=l 


Proof 


life  -  £  ai%l|2  =  (b  -  £  aiSi  »  b  -  (1-99) 


i=l 


i=l 


i»l 


=  (h,h)  -  £  ai(gi,b)  -  ^ai(b»Si) 


i=l 


i=l 


n  n 

+  Z  Z  aiaJ  (§i'V 

i=l  J=1 


n 

=  (b,h)  -  £  ai(ei,b)  - 
i=l 


n 


Z 

i=»l 


i=l 
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n 

By  adding  and  subtracting  the  expression  ^  (s^,h  )(h,  e^ )  to  the  right 
side,  one  finds 

n  n  n 

life  -  y  aiei  ||2  =  (fc,h)  -  y  Uh,^)!2  +  ^  1^"  (!?>%) 

i=l  i=l  i=l 

The  first  two  terms  of  the  right  side  do  not  depend  on  the  coefficients 
a^;  therefore,  the  minimization  of  the  left  side  expression  requires 
that 

ai  =  (h,^)  i=l,2,...,n  (1-100) 


Furthermore,  the  solution  to  this  problem  of  least  square  approximation 
is  unique,  and 


minimum 

a. 


lb  -  £  ..5/  .  tell2  -  £ 


10?,%)  | 


(1-101) 


i=*l 


i-1 


This  is  the  square  of  the  distance  &  of  the  point  h  to  the  linear  manifold 
spanned  by  an  orthonormal  set  ^e^,  e^,  . .  .,  e^  (  .  Since  the  minimum  of  a 

distance  cannot  be  a  negative  quantity,  we  have 
n 

y  l(fe,si)l2  <  llhll2  d-ioe) 

i=i 

This  inequality  is  referred  to  as  Bessel's  inequality.  The  preceding 
theorem  can  be  generalized  to  the  case  of  inf inite- dimensional  Hilbert 
space. 

Let  =  [ep^,...,?n,...J  be  a  complete  set  of  orthonormal  elements  of 

a  Hilbert  space  H,  and  L  the  linear  manifold  generated  by  the  first  n  vectors  of 
.  For  any  given  element  h  and  specified  e  >  0,  we  can  approximate  h  by 
a  linear  combination  of  elements  of  L 
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such  that 


E  ai%‘ 


||h  -  ^  <  e  for  n  >  N> 


some  N 


That  is,  this  representation  converges  in  the  metric.  The  lea3t  value  of  the 
expression  to  the  left  is  obtained  by  the  so-called  Fourier  coefficients,  i.e., 
when 

a.  =  (h,£.)  i-1,2,  ...,n  (1-103) 

1  ±  OO 

We  now  show  that  the  series  >  a,  e  converges.  Let  s  and  s  be  respectively 

•  *  K  ~~p  q 

k=l 

the  sum  of  the  first  p  and  the  first  q  terms  of  the  Fourier  series.  For  q  >  p 


we  will  have 


Is  -  si, 

q  p 


E  =  E  lakr 

p+i  p+i 


(1-104 ) 


As  p  and  q  are  increased  infinitely  the  sum  ^  la^  |2  converges  to  zero. 

P+1 

Therefore,  the  sums  s^  form  a  fundamental  sequence  in  the  H-space.  Let  .g  be 
the  limit  for  the  sequence.  Since  the  space  H  is  required  to  be  complete,  the 
point  s  must  belong  to  H.  The  fact  that  h  =  s  can  be  seen  from  the  following. 
In  view  of  the  continuity  of  the  inner  product,  the  Fourier  coefficient  (h,^) 
can  be  obtained  as  follows: 

P 

(s,e  )  =  lim  (s  ,e  )  =  lim  (  >  a,e.,e  )  (1-105) 

m  ^p'^m  '  /  i—i'-m' 

n  — »  do  _»  oo  ^ — ‘  -  “ 


=  a  =  (h,  e  ) 
m  ~m 
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For  any  fixed  m  and  p  >  m,  we  find 


(h-s*^)  =  (£>§m)  -  (s,^)  =  0  (1-106) 

The  identity  of  )i  and  s  follows  from  this  relation  and  the  fact  that  the  set 
£ej  is  complete.  Whence, 

00 

h=  lim  s  =  V  an£n  (1-107) 

P  -»  00  £3. 


Theorem 


Let  H  be  a  separable  Hilbert  space,  with  a  complete  set  of  orthonormal 
vectors  Sq> -Sj,  •  •  •  3  *  We  can  associate  with  any  element  x  e  H  a 

Fourier  series  (x,gk)  converging  in  the  norm  to  jc,  and  such 


k=l  00 


that  the  series 


y  |(x,ek)|2  converges  to 
k=l 

y  l(»*ek)|2  =  (x,x) 


(1-108) 


k=l 


Alternatively,  we  state  that  every  vector  t  H  can  be  approximated  as  closely 
as  desired,  in  the  mean  square  sense,  by  the  above  Fourier  series.  An  element 

OO 

can  be  exactly  expressed  by  a  Fourier  sum  ak^k  Besse^-'s  inequality  is 

k=l 

changed  into  the  so-called  Parseval's  equality;  that  is. 


y  l(£,Sk)|2  =  (s,2<)  (1-10$) 

k=l 

Conversely,  one  may  state  a  criterion  for  completeness  of  a  set  of  orthonormal 
vectors  in  a  Hilbert  space  as  follows: 
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Theorem 


In  order  that  the  set  of  orthonormal  vectors  e  be  a  complete  set, 
it  is  sufficient  that  Parseval's  equation  holds  for  every  x  e  H. 

OO 

(x,x)  =  Itell2  =  ^  l(x,ek)|2 

k=l 

If  H  is  also  a  complete  space  (as  in  our  present  case),  then  this 
condition  is  necessary  and  sufficient  for  completeness  of  £e^ 

In  case  of  the  space  L2[a,b],  any  arbitrary  nonzero  vector  f(t)  may  be 
represented  in  terms  of  its  Fourier  expansion  as: 

OO 

f(t)~  Z  akek(t) 

k=l 

But  this  representation  does  not  necessarily  converge.  However,  when  Jek(t)j* 
is  a  complete  orthonormal  set,  then  it  is  well  known  that  the  series  will 
converge. 

In  the  spirit  of  the  elementary  scope  of  this  chapter  we  have  primarily 
considered  separable  Hilbert  spaces,  which  are  obtainable  as  a  simple  extension 
of  the  Euclidean  space.  In  reality,  Hilbert  space  may  have  countable  dimensions. 
Then  the  left  side  of  the  Bessel  inequality,  Eq.  (1-102),  will  represent  an 
uncountable  set.  The  impact  of  such  a  generalization,  on  the  selection  of  a 
complete  orthonormal  set,  and  related  convergence  consideration  for  Fourier 
expansion,  will  require  a  full  understanding  of  Lebesgue  integral;  this  is 
beyond  our  plans . 

_ 

The  completeness  of  a  metric  space  X  and  the  completeness  of  a  set  of  ortho¬ 
normal  vectors  in  X  are  to  be  distinguished. 
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In  brief,  as  long  as  we  remain  content  with  a  direct  countable  infinite¬ 
dimensional  generalization  of  the  Euclidean  space,  things  are  under  control. 

A  step  beyond  that  into  the  most  general  type  of  Hilbert  and  Banach  space 
requires  more  specialized  mathematical  preparations. 

1-15  Representation  by  Fourier  Tr igonometr ic  Polynomials  in  a  Function  Space 
In  this  section  we  will  apply  the  content  of  the  previous  section  to  a 

problem  of  common  interest.  Let  h(t)  be  an  arbitrary  vector  of  the  real  function 

2 

space  L  [0,2k].  It  is  desired  to  obtain  the  best  Fourier  representation  of  h(tj, 
in  the  sense  of  minimizing  the  norm  of  error,  in  terms  of  real  trigonometric 
polynomials  of  degree  n  or  less: 

a 

f  (t)  =  ;r—  +  a,  cos  t  +  b,  sin  t  +  ...  +  a  cos  nt  +  b  sin  nt  (l-UO) 

n  d  1  1  n  n 

The  totality  of  these  polynomials  forms  a  subspace  S  of  dimension  2n+l  over  the 

field  of  real  numbers. 

We  set  forth  to  derive  the  trigonometric  polynomials  fn(t)  which  is  the 
best  linear  approximation  to  h(t )  in  the  least  square  sense,  that  is  minimizing: 

2n 

!|h-fnlf'  =  Y,  [h(t)-fn(t)f  dt  (1-111) 

0 

In  other  words,  the  problem  is  that  of  projecting  h  on  S.  Let  us  choose  a 
normalized  version  of  the  common  Fourier  tr  igonometr  ic  series  as  the  maximal  set 
le  lof  an  orthonormal  basis  for  S, 

1  _  cos  t  _  sin  t 

g°=  y^T  '  ^T7  '  "2  =  VT  ' 

(1-112) 

cos  nt  sin  nt 

=  V7-  '  e2n  =  VT“ 
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It  is  to  bo  noted  that  the  orthonormal  set  ^e^  is  "complete”,  that  is,  no 
additonal  non  zero  vector  of  S  can  be  found  which  is  linearly  independent  of 
the  elements  of  ^e^  •  Whence, 

2  n 

Prog,  h  on  S  =  I  ck% 


k=0 


where 


Ck  = 


(1-11M 


Recalling  the  usual  definition  of  the  inner  product,  we  find 

2n  2n 

c°  =  v%  /  h(t)  dt’  C2k-1  =  ^  /  h(t)  cos  kt  dt 


2k 


2  jt 

y7  l 


h(t )  sin  kt  dt 


(1-115) 


These  constants  specify  the  best  trigonometric  polynomial  approximation  for 

p 

elements  of  L~[0,2jt]  relative  to  the  $e  (  basis.  Since  the  question  was  formulated 

relative  to  f  ,  from  these  relations  we  derive  the  familiar  Fourier  series  co- 
~n 


efficients: 


dt 


(1-116) 


=  A  c_,  ,  =  -  /  h(t)  cos  kt  dt 

-y  it  2k- 1  jt  J 

v  0 

2  jt 

bk  -^CP.K  -  n  /h(t)  Sin  kt  dt 


(1-117) 


The  square  of  the  error  in  approximating  h  by  f  is: 
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(1-118) 


2n 

Ife-fJI2  =  fell2  -  Y.  I2 

i=0 

No  other  trigonometric  polynomial  of  the  above  type  fm(t)  with  m  <  n  may  offer 
a  lower  error.  When  n  is  increased,  the  error  is  reduced  and  when  n  -» 00  the 
converging  Fourier  series  expansion  approaches  the  exact  representation  of  h. 
Example  1-9 

Calculate  the  Fourier  coefficients  for  the  function 

f(x)  =  x  -it  <  x  <  JT 

in  [-it, it]  using  the  orthonormal  set 


% 


sin  x 

VT 


Solution: 


it 


The  projection  of  £  on  the  linear  manifold  of  the  above  orthonormal  set  is: 


2  (sin  x  - 


sin  2x 


+  si^x  _ 


2  3 

This  orthonormal  set  is  not  a  basis  for  the  infinite  dimensional  space.  A 
complete  set  of  orthonormal  basis  containing  the  above  set  can  be  obtained  by 
the  addition  of  elements 


cos  x  cos  2x 


'  VT  '  V2T  '  "I 
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But  the  projection  of  x  on  the  latter  set  is  the  vector  Q.  Therefore,  the 
exact  representation  of  x  in  the  infinite-dimensional  space  is 


x  =  2 


(-D 


k-1  sin  kx 


k=l 


1-16  Optimality  and  Closed  Convex  Sets 

A  set  of  points  X  in  a  vector  space  is  said  to  be  convex  if  for  any 
arbitrary  pail  3  x  and  j  of  X  and  0  <  t  <  1  we  have  z  =  [(l-t)x  +  tj]  c  X.  A 
simple  geometric  interpretation  of  this  property  may  be  visualized  by  thinking 
that  all  points  z  on  the  line  segment  connecting  £  and  y  and  between  £  and  y 
must  belong  to  X.  The  figure  below  sketches  a  non-convex  set  of  points: 


Fig.  1-8  A  non-convex  set  of  points 

In  view  of  this  definition  the  following  important  theorem  may  be  proved. 

Theorem 

Every  non-empty  closed  convex  set  X  in  a  Hilbert  space  H  contains  a 
unique  elements  h^  of  smallest  norm. 

Proof 

Let  6  =  inf  ||x||  for  x  e  X  denote  the  least  value  Of  ||x||  for  all  points 
in  the  convex  set  X.  Consider  a  sequence  of  points  in  X  such  that  ||xnll  -»  6 

and  n  -+ 00  .  A  proof  of  the  theorem  requires  the  following  two  steps: 

1.  ^x^is  a  Cauchy  sequence. 

2 .  x  — >  h_  and  n  — »  00 . 

-n  ~0 
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For  any  arbitrary  points  x  and  y  of  H  in  view  of  the  existence  of  an  5  nner  pro¬ 
duct,  the  so-called  parallelogram  law  gives: 

llac+iflP  +  llx-icll2  =  2\\xf  +  2\\xf  x,y  e  H  (1-119) 

i  h-xf  =  |  llxll2  +  |  hf  -  |fi^  II2  (1-120) 

Now  by  considering  points  x,  y  in  X,  in  view  of  the  convexity  of  X,  the  vector 
x+y 

will  be  also  in  X,  whence: 

llx-yll2  <2||jc||2  +  2  ||y iP  -4&2  x,y  e  X  (1-121) 

This  inequality  may  be  applied  for  points  xn  and  for  showing  that  ixn-  is 
a  Cauchy  sequence: 

llxn-xjl2  <  2|[xnl|2  +  ?hmf  -  4b2  (1-122) 

As  n  and  m  -»  °°  the  right  side  of  this  inequality  by  definition  tends  to  zero. 

Therefore  \x  <  is  a  Cauchy  sequence.  The  limit  point  of  this  sequence  must  be 
C~n  .> 

in  H  since  H  is  a  complete  space.  Let  h^  be  the  limit  point  of  the  sequence 
then 

||£n-h0ll  -♦  0  as  n  — >  °°  (1-123) 

The  point  h^  belongs  to  X  since  X  is  assumed  to  be  a  closed  convex  set.  It 
remains  to  show  that  ||h_ ||  =  lim  ||xnH  =  i.e.,  the  norm  is  a  continuous 

function.  To  prove  this  consider  once  again  the  fundamental  triangle  inequalities: 

lix||  <  ||x-y||  +  ||y||  (1-124  ) 

llxll  -  llxll  <  llx-xll  (1-12?) 

If  x  approaches  y,  the  right  side  tends  to  zero.  Thus,  ||x||  is  indeed  a  continuous 
function  on  H,  whence 
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(1-326) 


IlfeJI  -  11a  ,  llx  ll  -  6 

n 


TO  show  that  J}q  la  unique,  lat  l^ji  ■  ||j|| 
yields: 


\-tf  <  O 


*>0  “  * 


6,  then  the  Inequality  (1-320) 


(1-327) 


The  importance  of  this  theorem  ia  problems  of  application  la  self- evident. 
Whenever  ve  have  a  closed  convex  set  X  In  a  Hilbert  apace,  we  can  find  a  unique 
element  of  smallest  norm.  If  points  In  X  portray  the  performance  of  a  physical 
system  then  ^  describes  the  most  "efficient  performance"  among  all  g  <  X. 

Many  problems  of  optimal  control  fall  in  this  category.  The  challenge  to  the 
reader  is  to  identify  such  a  clear  mathematical  model  In  the  heart  of  the 
tedious  technical  literature  of  application  where  no  main  path  is  visible. 


Fig.  l-9a  The  element  of  Fig.  1-S*>  The  elements  of 

minimal  norm  is  unique  minimal  norms. 
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Chapter  2 


LUIXAR  OKRATCRS  Ilf  HUBERT  SPACE 

2-1.  Introduction 

In  th«  context  of  engineering  application,  the  material  of  the  pre¬ 
ceding  two  chapters  offers  a  representation  and  an  approximation  theory  for 
signals  in  a  Hilbert  space.  In  a  similar  vein,  the  material  of  this  chapter 
is  directed  toward  a  representation  theory  for  linear  processing  systems  of 
a  Hilbert  signal-space. 

The  theory  of  operators  is  a  powerful  tool  of  functional  analysis  with 
broad  applications  to  problems  of  engineering  and  physics.  For  example,  in 
control  theory,  wave  propagation  and  in  quantum  mechanics  one  is  constantly 
faced  with  the  problem  of  determining  critical  frequencies  of  a  physical 
system  which  amounts  to  the  determination  of  eigenfunctions  of  certain 
differential  or  integral  equations.  In  most  applications,  of  this  type  the 
concept  of  linear  operator  plays  a  very  important  role.  Take  for  instance  the 
familiar  integral  transformation  f (x)  -*  g(x ) 

b 

g(x)  -  f  k(x,y)  f(y)  dy  (2-1) 

Ja 

The  correspondence  between  functions  g(x )  and  f(x)  can  be  denoted  by  the  coat- 
pact  notation 

g  -  A  £  (2-2) 

In  this  format  one  can  readily  appreciate  that  the  solution  to  the  equation 

b 

f(x)  -  X  J  k(x,y)  f  (y )  dy  (2-5) 

a 

where  f(x)  is  an  unknown  function  is  coincident  with  solving  an  eigenvalue 
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problem,  since 


t-XA£ 


<a-4) 


UkM lM|  the  solution  to  the  vector  differential  equation 

i  -  A  «  *(0)  -  4  (2-5) 

where  A  la  a  constant  n  x  n  matrix,  was  found  to  be 

«  -  «At  «  (2-6) 

we  have  discussed  In  detail,  that  the  solution  vector  can  be  obtained  from 
the  Initial  condition  vector  4  by  applying  the  linear  operator  e£*. 

Sane  of  the  basic  properties  of  linear  operators  on  finite- dimensional 
vector  spaces  were  discussed  In  Chapter  3  of  TR-65-39b.  The  Information  pro¬ 
vided  there  was  adequate  for  an  introductory  treatment  of  transformation  rf 
elements  of  a  finite  dimensional  euclidean  space.  More  general  end  interesting 
results  may  be  derived,  however.  If  the  content  of  that  chapter  is  extended  to 
linear  operators  In  Hilbert  and  Banach  spaces.  The  fulfillment  of  this  aim  la 
the  Object  of  the  present  chapter. 

2-2.  Functionals  and  Operators 

By  direct  analogy  with  the  functions  defined  In  a  finite- dimensional 
vector  space,  one  can  define  scalar  and  vector  functions  over  the  elements  of 
a  linear  space  of  Infinite  dimension.  The  first  category  of  these  functions 
is  commonly  referred  to  as  functionals  and  the  second  type  as  operators. 

Let  D  be  a  subset  of  the  space  X,  to  each  point  {  c  D,  one  may  attach  as 
scalar  valued  function  ^(£).  This  is  a  functional  whose  domain  is  D. 

The  definition  of  an  operator  in  an  Infinite-dimensional  space  Is  essentially 
the  same  as  in  the  case  of  a  finite-dimensional  space.  Let  X  and  Y  be  two 
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i  , . -'.a.  ,,l. 


linear  spaces,  and  consider  the  aat  D  <=  Xj  to  every  aloaMnt  ]  «  1,  by  i 
suitable  rult,  we  associate  one  element  y  (  T.  The  relationship  between 
g  end  x  wey  be  designated  by 

*  -  A  X  (2-7) 

Die  set  D  is  referred  to  as  the  InaMln  of  definition  of  the  operator  A.  Dm 
set  Da  of  all  corresponding  lnage  points  jr  Is  called  the  ranee  or  the  dossal  n 
of  values  of  the  operator  A.  Xf  the  doamln  of  definition  covers  the  entire 
space  X,  and  If  e  X,  then  It  Is  said  that  the  operator  A  amps  X  Into  Itself, 
or  A  Is  an  operation  on  X. 

In  the  following  ve  shall  concentrate  on  linear  operators  acting  an 
linear  named  spaces  In  general,  and  an  Hilbert  spaces  for  the  aost  part.  In 
this  respect,  first  ve  need  to  define  the  linear  operator,  and  then  fully  grasp 
the  weaning  of  the  two  Independent  concepts,  continuity,  and  boundedness  for 
linear  operators  of  a  Metric  space. 

2-3.  Linear  Operators 

The  definition  of  a  linear  operator  napping  an  Infinite- (linens local  real 
linear  space  X  to  Y  stands  essentially  the  sesM  as  In  the  case  of  a  real  finite 
dimensional  space.  There  la  one  important  difference  however  that  In  the  finite- 
dlnenalonal  case  the  requlresMnt  of  continuity  does  not  present  itself  explicitly. 
In  the  infinite- dimensional  case  an  operator  is  said  to  be  linear  If  It  Is 
additive  and  oontluuous  as  described  below. 

a)  Additivity,  for  every  pair  ^  «  X  ve  require: 

*(*!  +  aS2)-A*1  +  A*2  (2-8) 

b )  Continuity.  If  a  sequence  ^  t  X  converges  to  g  c  X,  then  we  require 

that  the  sequence  A^i  !  converges  to  A  g  c  Y  in  the  sense  of  the  convergence 
In  Y. 


Based  ob  properties  i)  and  b)  one  eon  shoe  that  a  linear  operator 
la  Ow  hononensoua*.  l.s.,  for  erery  real  aaber  A(\,g)  ■  K  A  g. 


for  any  arbitrary  additive  operator  we  bevei 
A(fi)  -  A <fl  ♦  $)  -  A(fi)  A(fi)  -  fl 
A<»)  ♦  A(-g)  -  AC*-»)  -  fl  ,  A(-g)  -  -A(g) 


(2-9) 

(2-10) 


If  X  and  T  are 


ipsoeii  the  additivity  as  described  In  iq.  (2-0) 


la  lasafflelant.  One  bee  to  add  the  regain 
A(i  j).1A  (*)  ,  1  -V^T 


We  shall  state  without  proofs  that  a)  every  additive  and  homogeneous  opes 
In  asy  finite  dlnsnal rial  nornsd  spaoe  is  a  linear  operator,  b)  Svery  II 
operator  is  homogeneous.  (See  for  Instance  B.Z.  VUllkht  Introduction  to 


(2-11) 

i  operator 


operator  is 


Functional  Analyses,  English  translation  by  X.V. 


Dress, 


Mew  Torfc  1963, 


In  general  It  Is  easier  to  cheek  the  hoasogenelty  of  an 


A  it  Mid  to  to  a  continuous  operator  at  x^  if  tba  ln»  of  tba  apbara  8  by 
A  raamlns  within  as  arbitrarily  Mall  apbara  8ft  of  tba  apaoa  1^ 

lAs  -  AJqI  <  «}  (2-V) 

la  otbar  words,  if  far  any  arbitrary  a  >  O  tbara  axiata  a  6  >  0  such  that 
tba  <T‘  of  all  polats  of  8  r— In  within  8^. 

An  oparator  ooatiauoua  la  a  domain  Should  poasaoa  this  proparty  for  *-U 
points  in  that  domain. 

Aa  additiwa  oparator  baa  aa  additional  intaraatiac  proparty  that  if  it 
is  continuous  it  any  aiacla  point  ^  <  X  than  it  will  ba  continuous  for  tba 
wbola  apaoa  1}  and  tharafara  it  la  a  linaar  oparator.  la  fact,  lat  |  ba  tba 
limit  of  a  aaquanoa  than  •  j  ♦  approaebaa  for  a  -*«. 

Tharafara; 

Afcfe  -  A  ♦  Mo)  -*AX0  (2-1*) 

AJfe  -  Al  ♦  AJ^^A^  (2-15 ) 

A*,,-** 

b)  BouadadnaM.  Aa  oparator  A  dafiaad  ovar  a  watrlo  apaoa  K  la  amid 
to  ba  boundad  if  tbara  axiata  a  oonataat  M  >  0  such  that 

lAsI  <  -  M  for  all  s  a  8  (2-16) 

A  gsomatrlc  "ramindar"  of  tba  ooaeapt  of  bouadadaaaa  la  akatotaad  in  ri<*.  2-2. 
Tba  norm  of  tba  iaapa  point  As  any  point  s  with  a  flnita  norm  aaaot  baeomc 
arbitrarily  Xarga.  la  otbar  words,  tba  1aaa»  of  any  spoelflad  apbara  Rg|  -  k 
moat  ba  eontaiaad  within  tba  apbara  BAxI  <  1*  of  tba  1^- apses.  Tba  diataaoa 
batwaaa  any  two  points  s  and  y  of  should  satisfy  tba  inequality 

lAS  -  Azl  -  |A(s-x)l  <  Nls-zl  (a-17) 
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An  operator  need  not  be  bounded.  For  instance,  the  operation  of  differentiation 
la  linear  but  it  nay  generate  unbounded  results.  LUtevise,  an  operator  need 
not  be  continuoua.  For  additive  operator  a,  however,  it  can  be  shown  that 
the  definition  of  continuity  and  boundedneaa  beoona  equivalent. 

If  an  additive  operator  ia  bounded,  then  it  la  eaay  to  show  that  it  la 
alao  continuoua.  In  fact,  consider  a  sequence  of  vectors  . . .] 

of  the  apace  1  converging  to  g.  Far  the  corresponding  sequence  of  lasgss,  we 
can  write: 

Hf*  -  A*JI  -  IIa(m*)U  <  M  HmJ  (2-lfl) 

share  M  la  the  constant  appearing  in  Kq.  2-3 6.  Thus,  the  sequence 
^g1,Agg,  .  . .  ,Agn,  .  .  ^  converges  to  Aj  ai  n  <♦  ».  The  converse  is  also  true* 
a  continuoua  additive  operator  is  bounded.  (The  proof  of  this  stataawnt  la 

ainpia  and  is  available  in  no  at  texts  on  the  subject.  )  The  essence 
of  the  proof  la  that  when  a  vector  g  la  changed  continuously,  the  vector  At 
varies  also  continuously,  and  a  suitable  constant  K  >  O  nay  be  found. 

e)  Kara  of  a  Linear  Operator.  The  foregoing  arterial  suggests  a 
natural  definition  for  the  "nor*"  of  a  bounded  linear  operator  A  in  a  aetrlc 
apace  I.  The  norm  of  on  operator  A  la  the  eealleat  nunber  M  satisfying  the 
inequality  (2-18).  iron  this  definition,  it  follows  that  far  a  linear  bounded 
operator  and  any  arbitrary  vector  g,  we  have 
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(2-19) 


iu*  it  <  iuii  iy 

Another  equivalent  definition  of  ||a|I  la  as  follows: 

Wo  Mgr  up  the  closed  unit  sphere  of  the  space  S  by  the  linear 
operator  A,  an 1  search  for  the  supreaam  of  the  "length"  of  the  laage  point 
llAsI  In  K^<  The  nora  of  the  operator  A  la  denoted  by  IUII  and  defined  by 

IUII  -  aup  Ik* II  for  fell  <  1  (2-20) 

2-9.  The  Space  of  Linear  Qperatore 

We  have  already  shown  that  the  linear  operators  defined  over  a  linear 
vector  space  of  finite  dlasnsiona  theas elves  fora  a  linear  vector  space.  The 
null  operator  is  the  aero  elsasnt  of  this  space.  The  sua  of  two  linear 
operators  and  the  product  of  a  linear  operator  by  a  scalar  are  well  defined. 
Thus  the  linearity  of  the  space  of  the  linear  operator  Is  easily  established. 
The  Introduction  of  a  nora  for  the  operator  allows  us  to  state: 

Theorea 

The  space  of  linear  operators  defined  over  a  noraed  linear  space  is 
s  noraed  linear  space. 
ft:  oof 

The  validity  of  axloas  of  a  aetric  space  follows  directly  frea  1,  2, 

9  below. 

1.  In  the  first  place,  we  have  associated  a  nora  with  every  eleaent  of  the 
space  of  linear  operators,  l.e.,  a  non-negative  nuabsr 

IUII  -  suplUxll  >0  for  Ibcll  <  1  (2-21) 

This  nuaber  Is  equal  to  zero  If  and  only  if  A  is  the  null  operator. 
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2.  ItVAlj  -  eup  |jXAx||  -  M  «up|lAx|  -  lx,  I  lull 


(2-22)  ) 


tar  llsll  <  1 

3.  IU+bII  -  ■upllAjs+Bsll  <  rip  lUcll  ♦  *upfocU  .  UaII  ♦  UbII  (2*23) 

tar  Ifatll  <  1 


In  addition  to  tba  ab ova  theorem,  on*  can  darlva  tba  following  alaple 

% 

inequality  tar  tba  norm  at  tba  product  of  two  linear  operator  a  A  ■ 

v  llAsli  -  IU2(A12)II  <  lUgll  lUjjll  <  tUgll  IUJ  h\\  (2-24) 

That  la, 

(U^H  <  IIaJ  llAill  (2-23)- 

The  identity  operator  playa  the  role  of  unity. 

Evidently,  the  operation  of  multiplication  of  linear  operatora  ia  aaaociative 
and  diatributive,  i.e.. 


A^(AjAj  )  —  (AjAg  )Aj 

A3  (Al+A2 )  “  *3*1  *  *3*8 
(Aj+Ag^  -  AjAj  ♦  *2*5 


(2-*6) 
(2-27) 
(2-28) 
.2 


If  operatora  A^  -  Ag  -  A,  than  tba  product  A^Ag  may  be  denoted  by  A  .  In  tbla 
manner  we  can  define  pawera  of  an  operator  A  and  note  that  for  any  poaitive 
integer  n 

iuni  <  tur  (2-29) 


Theorem 

The  apace  of  linear  operatora  which  tranaform  the  nomad  apace  I  into 
a  complete  named  apace  E^  la  ltaelf  a  Banach  apace. 

Proof 


One  baa  to  ahov  that  the  linear  apaqe  of  operatora  ia  a  complete 
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; _ _  :*  1  ' 


JiL. 


•p*o*.  If  . ..^ . .  .,AQ.  •  .^  is  *  converging  sequence  of  linear 

operators,  that  is  |Un-A^||  — >  0  (n,m - >  «),  than  for  any  arbitrary  element 

2  of  I  the  corresponding  sequence  of  images  will  also  converge 

lUgg-Aysll  -  ll(VV  *H  5  IUn-Ajl  lbs II  - >  o  (2-30) 


Thus,  oe  sequence  ^Ax  j  is  a  converging  ae  qua  nee  for  every  g  which  converges 
to  Ad*  Since  the  norned  space  is  complete,  then  the  limit  of  any  sequence 
in  will  be  contained  in  Ijj  but  it  is  not  obvious  that  A  also  belongs  to 
the  space  of  linear  operators. 

Ihe  following  three  steps  are  required  for  completing  the  proof. 


a) 

to) 

c) 


A  is  additive 

A  is  bounded 

A  -  lim  A^ 
n  ■■  ^  • 


A  proof  may  be  derived  by  the  interested  reader.  (See  B.Z.  VUlikh,  Chapter  8, 
or  M.  Davis,  A  first  course  in  Functional  Analysis,  Cordon  and  Breach  Co., 

Hew  York  1966,  pages  49-30). 

2-6.  The  Inverse  Operator 

In  order  to  give  a  precise  definition  of  the  inverse  operator,  consider 
the  operator  A  which  maps  a  Banach  space  I  into  a  Banach  space  X . 

Z  -  X  *  I*  X  <  ^  (2-51) 

A  is  said  to  be  an  invertible  operator,  if  for  every  el  ament  ycl,  9u 
equation  j  ■  A|  has  a  unique  solution  g  in  B.  The  operator  which  represents 
this  correspondence  is  referred  to  as  the  inverse  operator  A  and  is  denoted 
by  A'1.  When  A  has  an  inverse,  we  can  formally  writ* 
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(2-J2) 


«  -  A"1  X 

AS  -  A  A-3*  -  X 

The  following  theorem  can  he  proven  without  difficulty: 

Theorem 

If  A  is  a  hounded  linear  operator,  whole  inverse  A-1  exists,  then 
A-1  is  also  hounded. 

The  proof  far  this  theorem  is  rather  long. 

It  requires  several  preliminary  steps  and  more  apace  than  we  can  afford.  The 
interested  reader  ia  referred  to  standard  mathematical  texts,  far  instance 
Kolmogorov- Fcsain,  Chapter  III.  t 

To  give  an  example,  note  that  the  operator  Ax  -  J  x(s)  ds  on  C[0, 1] 
to  C[0,1]  is  hounded,  but  A-1  y  «  y(t)  is  unbounded  °far  a  certain  subset 
Of  continuous  functions.  The  inverse  operator  does  not  exists  for  all  points 
of  the  space. 

Likewise  the  Sturm- Li ou villa  operator 

*>-!e  {r<‘>  *«<*)* 

which  is  defined  an  the  subspece  of  twice  continuously  differentiable  elements 
of  C[0,1]  is  unbounded.  Its  inverse  however  is  a  hounded  linear  operator  for 
all  C[0,1]  (Green's  function) 

1 

A-1  y  -  J  G(t,a)  y(s)  ds 
o 

In  dealing  with  operators  we  frequently  need  to  apply  the  following 
important  theorem  which  allows  a  power  series  expansion  of  the  inverse  (i-A) 
in  terms  of  powers  of  A. 
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Theorem 


If  A  is  an  operator,  with  ||a||  <  1,  mapping  a  Banach  space  E  into 
itself,  and  I  the  identity  operator,  then  the  inverse  of  the  operator 
I -A  can  be  written  as 
00 

(I-A)-1  -  £  A*  (2-33 ) 

k-0 

Proof 

Hie  proof,  in  essence,  is  similar  to  the  elementary  proof  for  convergence 
of  the  scalar  series 

00 

XIJ  -  £  x*  far  |x  |  <  1  (2-34) 

k-0 

Here,  we  consider  the  transformation  (1-AXjc  -  y,  and  set  forth  t.i  find  its 
solution  by  means  of  iterations: 


2n+l  "  **n  "  x  n  “  x»z>  • ♦  •  (2-55 ) 

With  »  x  this  procedure  yields: 

-  X  +  AX  +  A2y 

^  (2-36) 

2  p,  i 

J£n-JC  +  Ax  +  Ay.+  ...  +  A  ~  x 

Hie  key  to  the  convergence  of  this  sequence  is  the  fact  that  if  |{A II  <  q  <  1, 

then  ||An||  <  qn.  Therefore,  as  n  - >  °°,  yn  tends  to  the  unique  solution  of 

£  -  x  +  Ajc,  that  is, 

OO 

jc  -  Aky  (2-37) 

k-0 


-6l- 


Thus  ^  satisfies  jt  -  x  *  A £,  and  we  have  shown  elsewhere  (section  on  contraction 
mapping)  that  this  equation  has  a  unique  solution;  whence 


(i-a)-1*  -  y. 


AkX 


(2-38) 


k-0 


Example  2-1. 

Is  the  differential  operator  A  on  the  set  of  continuous  functions  in 
[0,1]  a  bounded  operator?  The  norm  of  x(t)  is  taken  as  max|x(t)|  in  [0,1]. 
Solution: 

AS  -  ft  [*(*>3 

Consider  a  sequence  of  points  on  the  unit  sphere: 


*  V  2  sin  nut 


n  v  1,2, ... 


Afn  -  V  2  nn  cos  nut 

llAjtJI  increases  without  bound  as  n  - >  00  ,  thus  the  operator  A  is 

unbounded.  Note  that  A  is  additive  but  not  continuous,  as  it  is  not  defined 
everywhere  in  the  space  C[0,1]. 

Example  2-2. 

Consider  a  mapping  of  the  linear  space  C[0,1]  of  all  functions  x(t ) 
continuous  in  the  interval  [0,1].  a)  Show  that  the  operator  A  defined  by  the 
integral  below  is  a  linear  operator  from  C  to  C. 

1 

y(t )  -  f  k(s,t)  x(t )  dt 
J0 

where  the  kernel  k(s,t)  is  a  function  continuous  in  the  square 


0  <  s  <  1 


,  0  <  t  <  1 
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b )  Show  that 


1 

||a||  -  max  f  |K(s,t )  |  dt  -  M 

0  <  s  <  1  ^0 

Solution: 

a)  Evidently  A  maps  C  Into  Itself.  Moreover, 

A(*+X)  -  Ajc  +  Ay 

The  continuity  of  the  operator  can  be  proved  by  considering  a  sequence  of 
vectors  jcQ  converging  to  Y:\  and  the  convergence  of  the  corresponding  sequence 
of  Images.  The  convergence  here  is  taken  In  the  sense  of  uniform  convergence, 
whence  we  can  take  the  limit  under  the  Integration  sign. 

b)  The  proof  Is  slightly  mare  complicated  than  a)  and  can  be  found  for  Instance 
In  Kantorovitch- Akilov' a  Functional  Analysis  In  Normed  Spaces  page  108-109. 

Example  2-3.  Determine  the  norm  of  the  following  operators: 

a)  Zero  Operator,  b)  identity  operator,  c)  similarity  operator,  A jc  » 

Solution: 

a)  ||a||  -  0, 

b)  I1a  II  -  1, 

c)  II A  ||  -  M 

Example  2-4.  (operators  of  the  normal  form).  Let  £g  ^  be  a  complete  ortho- 
normal  system  for  a  Hilbert  space  H  of  countable  dimension,  and  1\_ ,\j, . .  .,\  , . . 
a  bounded  sequence  of  real  numbers  with  |\^|  <  C.  For  every  x  ■  ^  xi 

of  H  we  define  the  following  operation: 

00 

**  -  T.  xj 
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a)  Is  A  a  linear  operator? 

b)  Find  |UII. 

Solution,  a)  A  is  a  well  defined  operator  on  H.  Its  linearity  is  an  immediate 
consequence  of  the  properties  of  additivity  and  homogeneity, 
b)  let  C  be  the  upper  bound  of  the  sequence  |\  |  -  C,  and  let 

oo  n 

lljcll2  »  £  xi  *■  1»  then 

i-1 

lUxll2  -  Y.  4  -  ^ 

i-l 

whence  |UI!  <  C 
On  the  other  hand 

lull  >  sup|Usnll  -  «up  II  -  eup  iunll  -  c 

Consequently  |U  II  *  C 

2-7.  Approximate  Solution  of  Functional  Equations 

Consider  the  linear  operator  A  defined  over  a  normed  space.  Frequently, 
we  wish  to  solve  a  functional  equation  of  the  type  AiS  -  x  ot*  3  «  Ajc  +  y 
for  a  given  y,  but  the  inverse  operator  A-1  is  not  known*  This  is  for  example 
the  case  when  it  is  desired  to  solve  a  set  of  algebraic  equations  but  the 
inverse  operator  A-1  (inverse  matrix)  is  not  readily  available}  or  when  an 
exact  solution  to  an  integral  or  differential  equation  is  hard  to  derive. 
According  to  a  method  suggested  by  L.V.  Kantrovicb,  A~^  may  be  approximated 
in  the  following  way. 
a)  IS  -  Ay  +  X 

Let  A  be  a  linear  operator  mapping  a  Banach  space  X  into  itself,  I  the 
identity  operator  of  X,  and  Aq  an  apprcKimant  to  A,  then 

(l-A)y  -  y  (I-A0)y0  -  x 
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The  success  of  the  method  e  xpends  on  the  choice  of  a  suitable  Aq  such 
that  we  can  assume  that  A-AQ  -  A  is  relatively  small,  that  is,  |!a||  ll(l-A)“1||<  1, 
and  if  (i-A)”^"  exists.  Under  these  conditions  it  follows  that  (I-Aq)  1  also 
will  exist  and: 

XQ  -  (I-A0)-1X  =  (l-Ao)-1(l-A)«  -(I-A0)_1  [(X-A0)+(A0-A)]at 
=  (l-Ao)"1(X-Aote  +(X-Ao)"1(A0-A)ic 
*o  “  *  -  (I-A0)_1(A0-Ate 

IUcoll  <  ll(X-Ao)-1||  ||A0-Ali  llacll  <  (I-A^"1  M  lljcll  (2-39) 

Thus  if  Aq  is  a  approximant  for  A,  then  for  any  y  the  functional  equation 
(l-Ao)aco  -  y  will  yield  an  approximate  solution  for  jt.  This  inequality 
however  does  not  explicitly  indicate  a  bound  for  the  error.  If  Aq  is  chosen 
such  that: 

llAll  ||(l-Aor1||  <  q  <  1 

Then  we  find: 

llxo-xl!  <  q  llxll  <  q(llx  -  xjl  ♦  lixoll)  -  dlx-xjl  +  qllxoll 

or 

•ISo-*11  ^  A  •M* 

to)  X  =  AjC 

Let  Aq  be  an  approximation  to  A  such  that  A”1  can  be  found  more  easily. 
Let  A  =  A-Aq,  and  assume  that  ||a||  |!ao  If  <  1.  For  a  given  y  we  may  approximate 
jc  by  the  element: 

Xc  -  A^y  (2-40) 
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Hie  error  vector  ^  satisfies  the  inequality: 


lls-*0ll  -  llA"1X-A0"aillA-1-A0“1||  lbr||  (2-41) 

It  can  be  shewn  as  an  exercise  (for  instance  by  analogy  with  the  scalar  case) 
that  under  the  above  hypothesis 


llA-1-A0-1|| 


<  Kl* 

i-  IIa0-1II  m 


(2-42) 


In  spite  of  the  fact  that  A-1  is  not  known,  this  inequality  provides  a  direct 
upper  estimate  for  the  norm  of  error,  by  writing 


lbs-js0ll  <  IIa-1-ao“1II  | he  II  < 


IIao-1II  the  II 
i  -  IIa0-1||  I|a|| 


(2-43) 


An  example  of  the  application  of  Kantorovich  method  is  to  be  found  in  solving 
Fredholm's  integral  equation,  by  replacing  its  kernel  with  a  degenerate  kernel. 

2-8.  Representation  of  Linear  Operators  in  A  Hilbert  Space 

In  Chapter  3  we  discussed  the  general  form  of  linear  operators  defined 
over  finite-dimensional  vector  spaces.  In  this  section  we  nake  a  generalisation 
of  that  material  for  linear  operators  on  Hilbert  and  Banach  spaces.  Here  we 
shall  investigate  whether  linear  operators  on  Hilbert  space  also  admit  a  matrix 
representation  similar  to  the  representation  of  Sec.  3-8  of  n 1-65-399. 

Consider  a  separable  Hilbert  space  with  an  arbitrary  complete  set  of 
orthonormal  elements  •  •  *4^*  •  •  •  J?  *  With  respect  to  this  set,  any  vector 

jc  e  H  can  be  uniquely  represented  as 

00 

*  “  H  Vk 

k-l 
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■where 


A  linear  (continuous)  operator  A  maps  £  to  x  *uch  that 

00 

x  »  As  -  £  *k  *®k 

k-1 


(2-1*5) 


(2-1*6) 


The  Image  of  e^  under  A  may  be  specified  by  its  Fourier  coefficients,  i .  e . , 


Ae^  "  “kj  £J  k  “  (2-1*7 ) 

>1 

Therefore,  the  Fourier  coefficients  of  y  in  its  representation  with  respect 
to  the  £  basis  are 


X  - 


Z  yj~ej 

j-i 


Z  % 


(2-48) 


whence  _ 

00 

yj  “  Z  V)  *k  *•*•*•  — 

k-l 


(2-1*9) 


In  this  manner,  the  vector  y  »  Ag  is  completely  specified  through  an  infinite 


matrix: 


*n 

*12 

...  a^ 

*21 

*22 

...  *2][ 

*di 

*J2 

...  aJk  ... 

_ 

(2-50) 
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There  is  an  important  difference  between  representation  theorem  of 

linear  operators  in  the  case  of  Hilbert  space  (and  B- spaces)  of  infinite  and 

finite  dimensions.  In  contrast  with  the  finite  dimensional  case,  not  all 

* 

infinite  matrices  represent  linear  operations  on  some  Hilbert  space. 

2-9.  Adjoint  Operator 

In  8ec.  (4-8)  of  IR-65-399  it  was  point  out  that  with  every  linear  operator 

A  defined  on  a  real  Euclidean  space  of  finite-dimension  Rn,  we  can  uniquely 

* 

associate  an  operator  A  in  a  manner  that  for  any  pairs  of  vectors  jc  and  y  of 
that  space 

(A*s,y)  -  ts,Ay)  (2-51) 

* 

The  operator  A  is  called  the  adjoint  of  the  operator  A.  In  any  orthogonal 

* 

basis,  the  matrix  representing  A  is  the  transpose  of  the  matrix  of  A  .  In 

particular,  it  may  occur  that  the  matrix  of  A  is  a  symmetric  matrix,  i.e., 

* 

alk  -  afc1 .  In  such  a  situation  we  have  A  »  A  ,  whence 

(Ajc,y)  .  (*, Ay)  (2-52) 

When  this  relation  is  satisfied,  the  operator  A  is  said  to  be  self-adjoint. 

When  the  space  is  a  complex  Euclidean  space  C11,  the  above  defining 

* 

equation  remains  valid;  but  the  elements  of  A  and  A  are  generally  complex 

numbers.  In  the  npecial  case  where  a^  -  then  A  is  identical  with  its 

* 

transpose  conjugate;  that  is,  A  -  A  is  self-adjoint.  A  self-adjoint  trans¬ 
formation  A*  -  A  defined  on  C11  is  represented  by  a  Hermit lan  matrix. 

The  above  definitions  remain  essentially  valid  for  Hilbert  and  Banach 
spaces.  Let  A  be  a  linear  operator  on  H,  whose  range  is  also  in  H.  According 

# 

The  necessary  and  sufficient  conditions  which  the  matrix  in  Eq.  2-30  must 
satisfy  are  given  for  instance  in  L.V.  Kantorovich  and  Q.P.  Akilov,  Functional 
Analysis  in  Wormed  Spaces.  The  Macmillan  Co.,  New  York  1964. 
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to  Chapter  3,  for  each  pair  of  vectors  of  H,  the  quantity  (Ajc,y)  defines  a 
linear  functional  on  H.  We  consider  the  equation 


(Ak,x)  -  0s>£*)  (2-53®) 

For  every  pair  of  y  and  y,  this  equation  provides  a  unique  y  e  H.  When  y 
# 

is  changed,  y  will  he  dunged  also.  Let  us  denote  their  relationships  by 
y*  -  A*y,  or 

(AjffX)  -  (lS,A*y)  (2-53b) 

* 

This  relation  characterizes  a  unique  operator  A  on  H  with  range  in 

H,  referred  to  as  the  adjoint  of  A.  It  can  be  shown  without  difficulty  (see 

.  * 

Liusternik  and  V.  Sobolev  Sec.  23.  )  that  the  adjoint  operator  A  of  a  linear 
operator  A  defined  on  H  is  indeed  a  linear  operator  and  ||A*||  -  ||a||. 

Hie  following  properties  of  adjoint  operators  are  easily  verified.  If 
A  and  B  are  linear  operators  over  a  Hilbert  space  H,  and  A  a  scalar,  then 


(A+B  )*  -  A*  +  B*  (2-54) 

(AA)*  -  AA*  (2-55) 

(BA)*  -  A*B*  (2-56) 

,  -1.*  ,  -1 

(A  x)  -  (A  )  A  if  A  exists  (2-57) 


(2-58) 


An  arbitrary  self-adjoint  operator  is  clearly  a  normal  operator. 
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(2-60) 


Unitary  Operator,  mis  Is  e  aubclaaa  of  normal  operators  satisfying 
the  additional  requirement 

AA*  -  A*A  -  I  (2-6l) 

It  Is  to  be  noted  that  for  unitary  operators  Eq.  2-60  reduces  to 

lUxll  -  llxll,  since  (Ajc,Ajc)  -  (jc,A*Ajt)  -  (x,y) 

A  rotation  In  Hilbert  space  Is  an  example  of  a  unitary  linear  operator. 
The  rotation  operator  A  maps  the  space  into  Itself  while  preserving  the  norm, 
that  is:  llAxIl  -  II  xli- 

As  an  application  of  the  foregoing  mateial,  consider  the  familiar 

2 

linear  Integral  operator  of  L  [a,b], 

b 

y(«)  -  J  K(a,t)  x(t)  dt  (2-62) 

a 

where  K(a,t)  is  a  continuous  kernel  function,  whence 
b  b 

I^Sjt)!2  ds  dt  <  +  «  (2-63) 

In  our  current  notation,  this  integral  equation  is  written  as  y  ■  A*.  me 
#  #  # 

adjoint  operator  A  now  may  be  introduced  by  y  -  Ay.  We  require  that  the 
equation  (2-64)  remains  satisfied.  It  is  not  difficult  to  show  that  this 
requirement  is  equivalent  to  choosing  the  integral  transformation  below  with 
the  new  kernel  as;  v 


y*(«)  -  f  K*(s,t)  y(t)  dt 

J 

(2-64) 

a 

K  ( s, t )  -  K<t,s) 

(2-65) 
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In  order  to  verify  whether  this  is  the  true  adjoint  of  the  linear  operator 
under  cone  ider  at  ion,  we  write: 


(2-66) 


(2-67) 


(2-68) 


(2-69) 


(2-70) 


A  comparison  with  Eq.  (2-52)  completes  the  validity  of  the  statement. 

The  above  linear  integral  operator  is  self- adjoint  if 

STSTH  -  K(t,s)  (2-71) 

When  the  kernel  is  a  real  function  s  and  t,  then  the  requirement  for 
self-adjoint  will  reduce  to  that  of  the  symmetry  of  the  kernel,  l.e.,  K(s,t)  > 
K(t,s). 
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2-10.  Positive  Operator  a.  A  self-adjoint  operator  A  is  said  to  be  a  positive 
operator  if  for  any  £  e  H,  (Ajc,  x )  >  0.  If  (Ajc,jc)  >  0  for  every  non-zero 
vector  of  1,  then  A  is  said  to  be  a  positive-definite  operator. 

An  arbitrary  positive  operator  A  has  a  unique  (self-adjoint)  positive 
square  root  B,  'that  is,  B2  =  A,  B  >  0.  An  operator  A^  is  said  to  be  greater 
than  an  operator  A,  if  A^-A^  is  a  positive  operator.  The  following  statements 
caul  be  directly  established: 

*  * 

1.  For  arbitrary  operator  A,  we  have  AA  >  0,  A  A  >  0.  This  is  in 
view  of 

(A*4jc,2c)  =  (Ajc, Ax)  >0  (2-72) 

9 

2.  For  arbitrary  A,  A'  >  O. 

3.  If  A  >  0,  then  for  any  positive  integer  n,  An  >  0. 

2-11.  Symmetric  Operator. 

In  common  mathematical  terminology  the  term  symmetric  operator  applies 
to  an  operator  A  which  is  additive,  homogeneous,  and  satisfies 

(Ajc,x)  -  (jc,Ax)  (2-7?) 

Based  on  this  definition,  a  symmetric  operator  needs  not  be  bounded.  Thus, 
a  self-adjoint  operator  is  a  bounded  symmetric  operator.  The  class  of  self- 
adjoint  operators  is  a  subset  of  the  wider  class  of  symmetric  operators. 

Examples  of  applications  of  symmetric  operators  occur  in  the  study  of 
Sturm- LiouviUe  equations: 

-  (P(t)  ff)  -  q(t)  X(t)  -  \  x(t)  (2-7*0 

where 
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p(t)  is  a  continuous  function  C^[a,b] 
q(t)  is  a  continuous  function  C[a,b] 

\  is  a  real  parameter  and  the  boundary  conditions  can  be  arbitrary.  We 

search  for  the  set  S  of  functions  x(t)  which  are  twice  differentiable  and 
2 

belong  to  L  [a,b].  By  rewriting  the  left  hand  side  of  this  equation  as  an 
operator  A  acting  on  jc  we  find: 

Ax  =  A  x  (2-75) 

The  operator  A  is  additive  and  homogeneous  but  not  continuous,  thus  unbounded. 
In  order  to  show  the  symmetry  of  A,  we  calculate: 

b 

(A£,x)  »  -  J  [(px')'  +  qx]  y  dt  ,  e  S 
a 


u  u 

-  J  (px' )f  y  dt  -  J 


y  dt  -  j  qx  y  dt 
a 

In  view  of  the  assumed  bounds  1  conditions  we  find: 


(2-76) 


b  b 


/  (px,)'  y  dt  =  <*x">  y  /  /  (px')  y’  dt  =  ~  J  x’(py') 


a  a 


(2-77) 


b  b 


=  -x(py ' )  /*/  x(py'  )'dt  =  J  x(py ' )'dt 


whence 


(Ajc,  y)  =  -  J  x [py’)'  +  q  y]  dt  =  (*,  Ax) 


(2-78) 
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This  establishes  the  symmetry  of  the  Sturm- Li ouville  operator.  The  importance 
of  this  result  in  solving  boundary  value  problems  of  this  general  type  is 
due  to  the  fact  that  symmetric  operators  possess  a  number  of  simple  pro¬ 
perties.  The  most  interesting  property  of  these  operators  pertains  to  the 
class  of  positive  operators  and  can  be  outlined  in  the  following  statement. 

If  A  is  a  symmetric  operator  and  (Ajc,  jc )  >  0  for  all  jc  in  some 

linear  subspace  of  a  Hilbert  space,  then  all  its  eigenvalues 

are  non- negative. 

For  proof,  let  X  be  an  eigenvalue  of  A  and  jc  a  corresponding  eigen¬ 
function,  and  observe  that: 

(Ajc,  jc)  =  (Xjc,  ac)  -  \(ac,  jc)  >  0  (2-79) 

2-12.  Projection  Operator 

We  have  discussed  in  full  detail  the  projection  of  a  point  g  of  a 
Hilbert  space  H  on  a  subspace  Sc  H.  Let  aSg  be  the  unique  point  called  the 

projection  of  at  on  S.  This  relationship  between  a;  and  £q  may  be  denoted  by 

Pac  where  P  stands  for  "projection  operator"  or  "projector".  The  following 
properties  of  a  projection  operator  of  a  Hilbert  space  are  easily  verified. 

1.  pac  lac  -  Pas 

2.  jc  e  S  and  Pjc  =  x  are  equivalent  statements. 

3.  aE  1  S  and  Pjc  =  Q  are  equivalent  statements. 

4.  lladl2  =  li'Psll2  +  llac-f^ll2  • 

5.  liPacll  <  ||x||.  This  inequality  follows  from  4,  above,  and  implies  that 
P  is  a  bounded  operator. 

6.  ||p|j  =  1  unless  S  is  reduced  to  null  element. 
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To  prove  the  latter  assertion,  let  £  e  S  with  ||jc|i  =  1.  Whence  ||p||  < 

llpjcll  ^  llacli  =  1.  Since  ||p||  =  sap  ||p^c II  for  Ujc||  <  1,  we  must  have  in  this  case 

||p||  >1-  On  the  other  hand,  assertion  5  along  with  the  definition  of  the 

norm  of  an  operator  implies  that  ||p||  <  1.  Therefore,  we  conclude  that  ||p||  =  1. 

7-  Projectors  of  Hilbert  space  are  positive  operators;  that  is, 

P(ac, x )  >  O.  In  fact,  we  will  show  that 

(I*,*)  =  (P2jc,jc)  >  0  (2-80) 


The  following  statement  is  an  interesting  theorem  about  projection 
operators  of  a  Hilbert  space. 

Theorem 

The  necessary  and  sufficient  conditions  for  a  linear  operator  P  to 
be  a  projection  operator  in  a  Hilbert  space  are  a)  the  operator  P  be  self-adjoint, 
and  b)  P  be  an  idempotent  operator;  that  is,  =*  P. 

Proof 

The  necessity  of  a)  and  b)  will  be  considered  first.  Let  S  be  a  sub¬ 
space  of  H,  g  and  y  arbitrary  points  of  H,  and  P  a  projector  from  H  to  S. 

Consider  vector  jc',  y' 


x  -  I)t  =  2' 

(2-8la) 

y  -  Fjr  =  y' 

(2-8lb ) 

jc*  1  S,  y'  |S 

(2-82  ) 

=  (*-£',  X'  +  ?x) 

(2-85) 

=  ?x) 

For  part  b),  we  note  that  for  any  g  e  H 


P(P5)  =  P(jc-ac')  =  I>5  -  Ps’  =  Ps 


(2-84) 
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To  show  the  sufficiency  of  requirements  a)  and  b ),  let  S  be  the  set  of 
all  elements  jc  of  H  such  that  their  Images  by  the  linear  operator  P  satisfy 
the  relation  Pjc  »  jc.  One  must  show  that  S  is  a  subspace.  To  this  end,  any 


element  jc  e  S  can  be  conveniently  written  as 

jc  =  Pjc  +  (iS- Pjc)  (2-85) 

Note  that  by  assumption  P(Pjc  )  =>  Pjc,  therefore  Pjc  €  S,  and  jc  -  £  S.  More¬ 

over,  for  any  two  arbitrary  points  jc,  y  e  S  we  have 

(as-Es,x)  -  (*,x)  -  (Es,x)  =  (z>x)  -  (a?,ii)  =  o  (2-86) 

That  is,  jc  -  I^c  ]_  S. 

2-13.  Completely  Continuous  Operators 

Uiere  is  a  natural  class  of  linear  operators  in  a  Hilbert  space  which 

is  very  similar  in  its  behavior  to  the  class  of  linear  transformation  in  a 

finite-dimensional  space.  These  are  referred  to  as  completely  continuous 

operators.  A  bounded  linear  operator  A  on  H,  in  order  to  be  completely  continuous, 

must  satisfy  the  following  property:  If  ^jc^  «oy  bounded  sequence  of 

vectors  (that  is,  there  exists  some  k  >  0  such  that  for  all  n,  lia^H  5  k)>  the 

sequence  must  contain  at  least  one  convergent  subsequence. 

Hie  definition  of  completely  continuous  operator  encompasses  the  concept 

of  the  so-called  "compact"  sets.  Thus,  the  above  definition  can  be  replaced  by 

* 

an  equivalent  statement  using  the  concept  of  compactness. 


A  set  8  contained  in  a  met^ic^  space  X  is  said  to  be  compact,  if  from  any  in- 

it  is  possible  to  select  a  subsequence 

to  some  limit.  (Hie  limit 
over  a  metric  space 
X  is  said  to  be  completely  continuous,  if  the  image  of  any  bounded  set  of  X 
is  a  compact  set  in  the  image  space  Y. 


a  set  o  contained  in  a  metric  space  x  is  said  to  De  compac 
finite  sequence  of  points  e  S  it  is  possible  to  selec 

fgpl  tZrfif  •  • « > gny*  • » •  j  ^  <;  il  <  ...,  convergent  in  X  to  so 
may  or  may  not“elong  to  S.  )  *An  additive  operator  defined 
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In  a  finite-dimensional  space  every  bounded  linear  operator  is 
completely  continuous  since  it  maps  bounded  sets.  But  this  is  not  necessarily 
true  in  infinite-dimensional  spaces.  Take,  for  instance,  the  identity 
operator  in  f2  space.  The  sequence  of  an  orthonormal  basis  £eQ^ is  certainly 
a  bounded  sequence  since  lisnl!  =  1  for  all  n.  But  the  sequence  Ieft  contains 
no  convergent  subsequence.  In  fact 

Ilgn-SjJI  =  2  for  n  /  m  (2-87) 

The  identity  operator  of  an  infinite-dimensional  Hilbert  space  is  not 
a  completely  continuous  operator.  Completely  continuous  operators  along  with 
self-adjoint  operators  are  the  simplest  and  the  most  common  type  of  linear 
operators  in  a  Hilbert  space.  The  relatively  complete  results  available  in 
problems  of  applications  are  generally  pertinent  to  this  category.  In  particular, 
if  a  completely  continuous  operator  of  a  Hilbert  space  is  also  self-adjoint, 
the  structure  of  the  operator  will  resemble  a  generalization  of  the  symmetric 
matrices  representing  finite-dimensional  self-adjoint  operators.  The  broad 
class  of  linear  integral  operators  with  continuous  kernel  on  0  <  s,  t  <  1. 

b 

y(s)  =  J  K(s,t)  jc(t)  dt  (2-88) 

a 

offers  a  most  common  example  of  completely  continuous  operators  on  the  space 

p 

L  [0,1].  For  an  introductory  treatment  of  completely  continuous  operators, 
theorems,  and  related  proof  see  Kolmogorov  and  Fomin,  Chapter  IV,  or  other 
standard  texts. 

2-14.  Completely  Continuous  Self-Adjoint  Operators 

In  this  section,  we  state  some  of  the  properties  of  completely  con¬ 
tinuous  self-adjoint  operators  A  in  H. 
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As  we  know,  an  eigenvalue  of  A  Is  a  number  X  such  that  there  exists  a 


non-zero  element  satisfying 


A«0  =  Nj£0  (2-89) 

Any  element  jc,  satisfying  this  equation  is  referred  to  as  an  eigenvector  or 
eigenelement .  The  totality  of  eigenvectors  associated  with  an  e  igenvalue  X 
form  an  eigensubspa^e  which  will  be  denoted  by  The  subspace  H'  of  H  is  said 

to  be  an  invariant  space  of  the  self-adjoint  operator  A,  if  for  jg  €  H'  we  have 
also  AS  e  H' . 

As  an  extension  of  the  material  of  Chapter  k,  of  TO-65-399  the  following 
assertions  for  every  element  of  H  and  every  completely  continuous  self-adjoint 
operator  A  are  maintained. 

1.  The  expression  (Ag,jc)  i s  real. 

(As,jc)  =  (ac,A2?)  -  (AS, 2)  (2-90) 


2.  The  eigenvalues  of  A  are  real. 


(A2,x) 
X  =  - 

(2*2) 


(2-91) 


3.  Eigensub spaces  and  corresponding  to  distinct  non-zero 
eigenvalues  are  orthogonal.  Let  &  s  y  e 


A2-*12  Ay=XgX 

Then 

(2>X)  **  ^  (A2,X)  -  ^  (2>Ai)  =■  (2»Y) 

Whence 

(2»X)  =  0,  jc  ly 


(2-92) 

(2-93) 

(2-94) 
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4.  The  operator  A  has  at  least  one  eigenvalue. 

5 .  The  operator  A  has  at  most  a  denumerable  set  of  eigenvalues. 

6.  The  spectrum  of  a  self-adjoint  operator  A  lies  entirely  in  the 
interval  [m,M]  of  the  real  axis,  where 

M  =  sup  (AS,2)  for  |jjc||  =  1 

(2-95) 

m  =■  inf  (Aj£,;jc)  far  |ljc||  =  1 

(Far  the  proof  of  these  latter  statements,  see  standard  texts.) 

The  main  motivation  for  the  study  of  completely  continuous  self-adjoint 
operators  stems  from  the  fact  they  are  the  natural  extension  of  linear 
transformations  of  a  finite-dimensional  Euclidean  space.  As  we  know,  fiinite- 
dimensional  Euclidean  spaces  are  complete  and  separable  -  and  all  linear  operators 
on  such  spaces  are  completely  continuous.  A  most  essential  property  of  com¬ 
pletely  continuous  self-adjoint  operators  is  expressed  by  the  following  funda¬ 
mental  theorem  due  to  D.  Hilbert: 

Theorem: 

In  a  complete  separable  Hilbert  space  every  completely  continuous  self- 
adjoint  operator  possesses  a  complete  orthogonal  system  of  eigenvectors.  If 
the  proof  of  this  theorem  requires  more  space  than  what  is  available  at  present. 

In  view  of  this  theorem  one  can  visualize  the  particular  simplicity  for  handling 
problems,  one  may  search  for  a  complete  system  of  eigenvectors  from  which  a 
complete  set  of  orthonormal  coordinates  may  be  constructed.  Thus,  every 

point  jc  c  H  will  have  an  image  representable  as: 

00 

a*“  T.  xi(zisi)~i 

i=i 

Where  Ai(i=l,2,  . . .  )  are  eigenvalues  of  the  operator  A,  e^i  =  1,2,...)  the 
corresponding  eigenvectors,  and 
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(£i'-eJ)  -  6iJ 


(i^  J  =  1*2,  ...  ) 


8^  =  1  6^j  =  0  when  i  ^  J . 

We  will  show  In  the  chapter  on  Integral  equations  that  due  to  the  avail¬ 
ability  of  orthogonal  coordinate  systems  in  the  function  space,  the  solution 
to  integral  equations  with  completely  continuous  self-adjoint  kernels  admits 
a  very  simple  form. 

♦Spectrum  of  a  self-adjoint  operator 
Consider  the  equation 

(A  -  XI)  y  -  y  (2-96) 

where  X  and  y  are  given,  y  is  unknown,  A  is  a  self-adjoint  operator,  and  X 
any  arbitrary  complex  number.  Let  us  assume  that  for  a  certain  value  of  X, 
the  operator  R^  -  (A  -  XI)"1  exists.  Then  R^  is  called  to  as  the  resolvent 
of  Eq.  2-96,  and  for  this  value  of  X  and  any  arbitrary  y  the  equation  (2-96) 
has  the  unique  solution  y  ■  R^  y. 

Those  X  for  which  Eq.  2-96  has  a  unique  solution  for  all  y  are  referred 
to  as  regular  values  of  the  operator  A.  Any  non  trivial  solution  of  the 
homogeneous  Eq.  2-96,  i.e.,  y  /  0  is  an  eigen  element  of  A  for  this  eigenvalue 
X.  The  totality  of  non-regular  values  of  X  is  called  the  spectrum  of  the 
operator  .A.  In  particular  all  the  eigenvalues  belong  to  the  spectrum. 

Example  2-U. 

a)  Show  that  the  (real)  operator 

A  y  =  tx(t)  y  c  L2[0,1] 

O  <  t  <  1 
is  self-adjoint. 

b)  Show  that  A  >  0 
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Solution: 


b) 


Example  9 

a) 

of  A. 
Solution. 

Hence 

(AC,  £)  > 


Clearly  A  is  an  additive  operator.  The  image  of  any  point 
2  2 

jc  e  L  is  also  a  point  in  L  .  Moreover  A  is  bounded  since 


Ml  <  fell 

Thus  A  is  a  linear  operator.  In  order  to  show  that  A  is 

self-adjoint  we  write: 

1 

(Ajc,x)  -  f  t  x(t)  y(t )  dt 


Jm 

(z>A*y)  =  J  x(t)  •  t(y(t )  dt 


(AS,x)  =  J  ttx(t)]2  dt  >  O 


It  can  be  also  shown  by  Cauchy- Buniakovski  ‘  s  inequality  that  for 

fell  =  1: 

0  <  <  1 

2 

-5-  In  the  function  space  L  [0,1]  an  operator  A  is  defined  by 
A  f  =  t  f (t ) 

Show  that  A  is  a  positive  operatory  b)  determine  the  square  root 


a)  For  two  arbitrary  functions  f(t)  and  g(t)  we  have 
1 

(A£>«)  =  ft  f(t)  g(t)  dt  =  (f,A*y) 

J0 

*  t  \ 

A  g  =  t  g(t),  that  is  A  is  self-adjoint.  We  note  also  that 
Oj  consequently  A  is  a  positive  operator. 
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b )  The  square  root  of  A  ia  defined  by  B: 


B  t  -  Vt”  r(t) 

Example  9-6.  Consider  the  operator 

t 

Ajc  -  f  x(a)  da 
Jo 

defined  over  the  Banach  space  of  real  contlnuoua  functions  C[0,1). 
a)  Is  A  bounded?  b)  Is  A-1  bounded? 

Solution:  a)  yes 

b)  A_1y  -  y(t) 

A*^-  la  not  bounded. 

Example  9-7*  Show  that  the  (real)  linear  operator 

Ajc  -  tx(t )  jc  c  l2  [0,1] 

0  <  t  <  „ 
la  self- adjoint. 

Solution: 

(Ajc.y)  -  t  x(t)  y(t) 

(jc,  A*y)  -  x(t)  •  t(y(t) 

Example  2-8.  Determine  the  eigenvalues  and  eigenfunctions  of  the  operator 

A  »  — — with  the  periodic  boundary  conditions 
dt 

x(0)  =  x(2jt)  ,  x'(0)-x'(2x) 
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Solution:  A  it  t  linear  symmetric  operator  on  the  aubapace  Cg[0 ,2*].  The 
determination  of  eigenvalues  amounts  to  solving 

x"  +  X  x  -  0 

t  -/x t 

x  ■  c^  e  +  Cg  e 

There  are  three  possibilities  to  be  exasiined: 

a)  X  <  O 

The  specified  boundary  conditions  requires  -  Cg  ■  O,  whence  there 
can  be  no  negative  eigenvalues. 

b)  X  -  O 

In  this  case  the  solution  to  the  differential  equation  reduces  to 
x  -  c^  +  Cgt.  The  boundary  conditions  impose  c2  -  O.  Therefore,  X  •  0  is 
an  eigenvalue,  and  all  functions  x  >  c1  in  [0,2s]  are  associated  eigenfunctions. 

c)  X  >  O 

In  this  case  one  finds  that  the  set  of  Integer sVVx"  -  1,2,3, .. .  are 

2 

eigenvalues.  With  each  eigenvalue  X^«k  ,  we  can  associate  an  eigenspace  of 
dimension  two  spanned  by  sin  kt  and  oos  kt. 

Example  2-9* 

-x"  «  t 

x(0)  ■  x(it)  ■  0 
d2 

Solution.  The  operator  A  -  -  — g  acting  on  the  subspace  C^[0,n]  satisfying 

dt  e  “ 

the  specified  boundary  conditions  is  a  linear  symmetric  operator.  The 
eigenvalues  of  A  maybe  found  from  the  equations: 

-x"  -  Xx  ■  0 
x(0)  -  x(n)  -  0 
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This 


2 

Tiitae  are  X.  =  n  ,  (n  =  1,2,...)*  The  set  of  eigenfunctions  £n=sinnt 
n  =  1,2,...  forms  a  complete  set  for  the  space  under  considerations, 
allows  us  to  write: 


(W 


f 


(x,fn)  =  J  t 


sin  nt  dt  =  (-l)n+^  «/n 


(fn,fn)  =  n/2 


x(t)  = 


V*  (-l)n+1  —  sill  nt 
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